# Encoding problem

**URL:** <https://www.ruby-forum.com/t/encoding-problem/201718>\
**Category:** IronRuby\
**Created:** [January 13, 2011, 10:39am UTC](https://www.ruby-forum.com/t/encoding-problem/201718 "2011-01-13T10:39:31Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![dubstep](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/cdc98d/32.png) [@dubstep](https://www.ruby-forum.com/u/dubstep)\
**Post date:** [January 13, 2011, 10:39am UTC](https://www.ruby-forum.com/t/encoding-problem/201718/1 "2011-01-13T10:39:31Z")

</div>

Hi Guys,

My boss thought it would be cool to use “” in an sql tablename, many of  
you will want to shoot her now J.

But now I did find something weird, I can’t even print “”.

It says:

tabaco.rb:16:in `puts’: character U+00EB can’t be encoded in US-ASCII  
(Encoding::InvalidByteSequenceError)

```
    from tabaco.rb:16

```

or

when I print the string somewhere else :S when it comes back from a  
method.

System::Text::DecoderFallbackException at /patient/0

Unable to translate bytes [EB] at index 3 from specified code page to  
Unicode.

Or when I don’t mess with it

Encoding::InvalidByteSequenceError at /patient/0

invalid byte sequence EB on UTF-8

All the same problem coming from 3 places.

Is this a fundamental issue or should this be solvable?

If you could point me in the right direction I could try to maybe fix  
it.

Thanks,

Albert-Jan

---

<div class="post-metadata">

**Author:** ![Albert-Jan\_Pieter\_N](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/76d3ee/32.png) [@Albert-Jan\_Pieter\_N](https://www.ruby-forum.com/u/Albert-Jan_Pieter_N)\
**Post date:** [January 13, 2011, 2:52pm UTC](https://www.ruby-forum.com/t/encoding-problem/201718/2 "2011-01-13T14:52:54Z")

</div>

Hi,

I don’t really know the solution to your question, but this might help:

is Unicode U+00EB, which is 0xC3AB in UTF-8 (so we are dealing with  
unicode rather than utf-8, which I assume is because IronRuby uses the  
immutable .NET strings internally with Unicode encoding).

The errors are expected if your default encoding is US-ASCII because it  
does  
not contain (and uses single bytes, so the 0x00EB would be broken into  
two  
bytes and your script would choke on the second 0xEB) : you will need to  
set  
your encoding to something compatible, like utf-8.

I don’t quite know how to do that properly in IronRuby, but in CRuby 1.9  
you  
could use “magic comments” in your ruby file and in 1.8 something like  
$KCODE=‘u’ could work. You might also be able to drop back into .NET and  
set  
the encoding there, but I’m not sure how that affects IronRuby  
assemblies.

I would start with $KCODE = ‘u’ Let me know how that works for you.

Zaki

On Thu, Jan 13, 2011 at 6:33 PM, Albert-Jan Pieter Nijburg \<

---

<div class="post-metadata">

**Author:** ![Albert-Jan\_Pieter\_N](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/76d3ee/32.png) [@Albert-Jan\_Pieter\_N](https://www.ruby-forum.com/u/Albert-Jan_Pieter_N)\
**Post date:** [January 13, 2011, 3:37pm UTC](https://www.ruby-forum.com/t/encoding-problem/201718/3 "2011-01-13T15:37:32Z")

</div>

Hey Zaki,

WARNING: YAML.add\_builtin\_type is not implemented

unknown:0: warning: variable $KCODE is no longer effective

tabaco.rb:11:in `puts’: character U+00EB can’t be encoded in US-ASCII  
(Encoding:

:InvalidByteSequenceError)

```
    from tabaco.rb:11

```

Too bad… thanks though. I’ll have a look in the source if I can find  
something.

Annoying Europeans 😛

Albert-Jan

Van: removed\_email\_address@domain.invalid  
[[mailto:removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)] Namens Dezso Zoltan  
Verzonden: donderdag 13 januari 2011 14:52  
Aan: removed\_email\_address@domain.invalid  
Onderwerp: Re: [Ironruby-core] Encoding problem

Hi,

I don’t really know the solution to your question, but this might help:

is Unicode U+00EB, which is 0xC3AB in UTF-8 (so we are dealing with  
unicode rather than utf-8, which I assume is because IronRuby uses the  
immutable .NET strings internally with Unicode encoding).

The errors are expected if your default encoding is US-ASCII because it  
does not contain (and uses single bytes, so the 0x00EB would be broken  
into two bytes and your script would choke on the second 0xEB) : you  
will need to set your encoding to something compatible, like utf-8.

I don’t quite know how to do that properly in IronRuby, but in CRuby 1.9  
you could use “magic comments” in your ruby file and in 1.8 something  
like $KCODE=‘u’ could work. You might also be able to drop back into  
.NET and set the encoding there, but I’m not sure how that affects  
IronRuby assemblies.

I would start with $KCODE = ‘u’ Let me know how that works for you.

Zaki

On Thu, Jan 13, 2011 at 6:33 PM, Albert-Jan Pieter Nijburg  
[removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

Hi Guys,

My boss thought it would be cool to use “” in an sql tablename, many of  
you will want to shoot her now J.

But now I did find something weird, I can’t even print “”.

It says:

tabaco.rb:16:in `puts’: character U+00EB can’t be encoded in US-ASCII  
(Encoding::InvalidByteSequenceError)

```
    from tabaco.rb:16

```

or

when I print the string somewhere else :S when it comes back from a  
method.

System::Text::DecoderFallbackException at /patient/0

Unable to translate bytes [EB] at index 3 from specified code page to  
Unicode.

Or when I don’t mess with it

Encoding::InvalidByteSequenceError at /patient/0

invalid byte sequence EB on UTF-8

All the same problem coming from 3 places.

Is this a fundamental issue or should this be solvable?

If you could point me in the right direction I could try to maybe fix  
it.

Thanks,

Albert-Jan

---

<div class="post-metadata">

**Author:** ![Albert-Jan\_Pieter\_N](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/76d3ee/32.png) [@Albert-Jan\_Pieter\_N](https://www.ruby-forum.com/u/Albert-Jan_Pieter_N)\
**Post date:** [January 13, 2011, 4:23pm UTC](https://www.ruby-forum.com/t/encoding-problem/201718/4 "2011-01-13T16:23:25Z")

</div>

Hi,

> warning: variable $KCODE is no longer effective

This means that you are in 1.9 mode 🙂 In that case there are two things  
you  
could try:

1. set the encoding at the top of the file in the form of the comment:

# encoding: UTF-8

1. force an encoding on the string(s) in question with the method (if 1)  
fails in IronRuby):  
.force\_encoding(“UTF-8”)

Zaki

On Thu, Jan 13, 2011 at 11:20 PM, Albert-Jan Pieter Nijburg \<

---

<div class="post-metadata">

**Author:** ![Albert-Jan\_Pieter\_N](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/76d3ee/32.png) [@Albert-Jan\_Pieter\_N](https://www.ruby-forum.com/u/Albert-Jan_Pieter_N)\
**Post date:** [January 13, 2011, 4:02pm UTC](https://www.ruby-forum.com/t/encoding-problem/201718/5 "2011-01-13T16:02:40Z")

</div>

I Just found this:

#Encoding:UTF-8

puts “patiënt”

which outputs: pati´┐¢nt

It doesn’t crash anymore J

Van: removed\_email\_address@domain.invalid  
[[mailto:removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)] Namens Dezso Zoltan  
Verzonden: donderdag 13 januari 2011 14:52  
Aan: removed\_email\_address@domain.invalid  
Onderwerp: Re: [Ironruby-core] Encoding problem

Hi,

I don’t really know the solution to your question, but this might help:

ë is Unicode U+00EB, which is 0xC3AB in UTF-8 (so we are dealing with  
unicode rather than utf-8, which I assume is because IronRuby uses the  
immutable .NET strings internally with Unicode encoding).

The errors are expected if your default encoding is US-ASCII because it  
does not contain ë (and uses single bytes, so the 0x00EB would be broken  
into two bytes and your script would choke on the second 0xEB) : you  
will need to set your encoding to something compatible, like utf-8.

I don’t quite know how to do that properly in IronRuby, but in CRuby 1.9  
you could use “magic comments” in your ruby file and in 1.8 something  
like $KCODE=‘u’ could work. You might also be able to drop back into  
.NET and set the encoding there, but I’m not sure how that affects  
IronRuby assemblies.

I would start with $KCODE = ‘u’ Let me know how that works for you.

Zaki

On Thu, Jan 13, 2011 at 6:33 PM, Albert-Jan Pieter Nijburg  
[removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

Hi Guys,

My boss thought it would be cool to use “ë” in an sql tablename, many of  
you will want to shoot her now J.

But now I did find something weird, I can’t even print “ë”.

It says:

tabaco.rb:16:in `puts’: character U+00EB can’t be encoded in US-ASCII  
(Encoding::InvalidByteSequenceError)

```
    from tabaco.rb:16

```

or

when I print the string somewhere else :S when it comes back from a  
method.

System::Text::DecoderFallbackException at /patient/0

Unable to translate bytes [EB] at index 3 from specified code page to  
Unicode.

Or when I don’t mess with it

Encoding::InvalidByteSequenceError at /patient/0

invalid byte sequence EB on UTF-8

All the same problem coming from 3 places.

Is this a fundamental issue or should this be solvable?

If you could point me in the right direction I could try to maybe fix  
it.

Thanks,

Albert-Jan

---

<div class="post-metadata">

**Author:** ![Albert-Jan\_Pieter\_N](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/76d3ee/32.png) [@Albert-Jan\_Pieter\_N](https://www.ruby-forum.com/u/Albert-Jan_Pieter_N)\
**Post date:** [January 13, 2011, 4:47pm UTC](https://www.ruby-forum.com/t/encoding-problem/201718/6 "2011-01-13T16:47:56Z")

</div>

Hey,

I found out that if I put nothing at the top and I do this:

puts “\x89”

it puts “”

if I put the #Encoding: UTF-8 at the top this happens. J

mscorlib:0:in `Throw’: Unable to translate bytes [89] at index -1 from  
specifie

d code page to Unicode. (System::Text::DecoderFallbackException)

```
    from mscorlib:0:in `Fallback'

    from mscorlib:0:in `InternalFallback'

    from mscorlib:0:in `GetCharCount'

    from mscorlib:0:in `GetCharCount'

    from mscorlib:0:in `GetChars'

    from tabaco.rb:2:in `puts'

    from tabaco.rb:2

```

It does print it but then it dies.

#\<Encoding: UTF-8\>

puts “\x89”.force\_encoding(“UTF-8”) does the same

#\<Encoding: UTF-8\>

puts “”.force\_encoding(“UTF-8”) does the same as before.

Also without the #

So I thought I had it with the puts “\x89” and I tried this:

class PatGeg \< ActiveRecord::Base

```
  set_table_name "Pati\x89ntGegevens"

```

end

PatGeg.first.Achternaam

and here’s what I got

c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby/gems/1.9.1/g

ems/activerecord-3.0.0/lib/active\_record/connection\_adapters/abstract\_adapter.rb

:200:in `log’: incompatible character encodings: UTF-8 and ASCII-8BIT  
(Encoding:

:CompatibilityError)

```
    from 

```

c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby

/gems/1.9.1/gems/activerecord-sqlserver-adapter-3.0.0/lib/active\_record/connecti

on\_adapters/sqlserver/database\_statements.rb:217:in `raw\_select’

```
    from 

```

c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby

/gems/1.9.1/gems/activerecord-sqlserver-adapter-3.0.0/lib/active\_record/connecti

on\_adapters/sqlserver/database\_statements.rb:178:in `select’

```
    from 

```

c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby

/gems/1.9.1/gems/activerecord-3.0.0/lib/active\_record/connection\_adapters/abstra

ct/database\_statements.rb:7:in `select\_all’

```
    from 

```

c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby

/gems/1.9.1/gems/activerecord-3.0.0/lib/active\_record/connection\_adapters/abstra

ct/query\_cache.rb:56:in `select\_all’

```
    from 

```

c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby

/gems/1.9.1/gems/activerecord-3.0.0/lib/active\_record/base.rb:467:in  
`find\_by\_sq

l’

```
    from 

```

c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby

/gems/1.9.1/gems/activerecord-3.0.0/lib/active\_record/relation.rb:64:in  
`to\_a’

```
    from 

```

c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby

/gems/1.9.1/gems/activerecord-3.0.0/lib/active\_record/relation/finder\_methods.rb

:333:in `find\_first’

```
    from 

```

c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby

/gems/1.9.1/gems/activerecord-3.0.0/lib/active\_record/relation/finder\_methods.rb

:122:in `first’

```
    from c:6:in ` __send__'

    from c:6:in `first'

```

I’ve tried to do the force\_encoding(“UTF-8”) on this thing to which  
results in something very similar :

c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby/gems/1.9.1/g

ems/activerecord-sqlserver-adapter-3.0.0/lib/active\_record/connection\_adapters/s

qlserver/quoting.rb:31:in `=~': invalid byte sequence 89 on UTF-8  
(Encoding::Inv

alidByteSequenceError)

```
    from 

```

c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby

/gems/1.9.1/gems/activerecord-sqlserver-adapter-3.0.0/lib/active\_record/connecti

on\_adapters/sqlserver/quoting.rb:31:in `quote\_table\_name’

```
    from 

```

c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby

/gems/1.9.1/gems/activerecord-3.0.0/lib/active\_record/base.rb:597:in  
`quoted\_tab

le\_name’

```
    from 

```

c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby

/gems/1.9.1/gems/activerecord-3.0.0/lib/active\_record/relation/query\_methods.rb:

234:in `build\_select’

```
    from 

```

c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby

/gems/1.9.1/gems/activerecord-3.0.0/lib/active\_record/relation/query\_methods.rb:

159:in `build\_arel’

```
    from 

```

c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby

/gems/1.9.1/gems/activerecord-3.0.0/lib/active\_record/relation/query\_methods.rb:

110:in `arel’

```
    from 

```

c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby

/gems/1.9.1/gems/activerecord-3.0.0/lib/active\_record/relation.rb:64:in  
`to\_a’

```
    from 

```

c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby

/gems/1.9.1/gems/activerecord-3.0.0/lib/active\_record/relation/finder\_methods.rb

:333:in `find\_first’

```
    from 

```

c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby

/gems/1.9.1/gems/activerecord-3.0.0/lib/active\_record/relation/finder\_methods.rb

:122:in `first’

```
    from c:6:in ` __send__'

    from c:6:in `first'

    from tabaco.rb:34

```

I have a feeling that ironruby and .net are not in sync with the  
encodings

Albert-Jan

Van: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
[[mailto:removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)] Namens Dezso Zoltan  
Verzonden: donderdag 13 januari 2011 16:04  
Aan: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
Onderwerp: Re: [Ironruby-core] Encoding problem

Hi,

> warning: variable $KCODE is no longer effective

This means that you are in 1.9 mode 🙂 In that case there are two things  
you could try:

1. set the encoding at the top of the file in the form of the comment:

# encoding: UTF-8

1. force an encoding on the string(s) in question with the method (if 1)  
fails in IronRuby):

.force\_encoding(“UTF-8”)

Zaki

On Thu, Jan 13, 2011 at 11:20 PM, Albert-Jan Pieter Nijburg  
[removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

Hey Zaki,

WARNING: YAML.add\_builtin\_type is not implemented

unknown:0: warning: variable $KCODE is no longer effective

tabaco.rb:11:in `puts’: character U+00EB can’t be encoded in US-ASCII  
(Encoding:

:InvalidByteSequenceError)

```
    from tabaco.rb:11

```

Too bad.. thanks though. I’ll have a look in the source if I can find  
something.

Annoying Europeans 😛

Albert-Jan

Van: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
[[mailto:removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)] Namens Dezso Zoltan  
Verzonden: donderdag 13 januari 2011 14:52  
Aan: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
Onderwerp: Re: [Ironruby-core] Encoding problem

Hi,

I don’t really know the solution to your question, but this might help:

is Unicode U+00EB, which is 0xC3AB in UTF-8 (so we are dealing with  
unicode rather than utf-8, which I assume is because IronRuby uses the  
immutable .NET strings internally with Unicode encoding).

The errors are expected if your default encoding is US-ASCII because it  
does not contain (and uses single bytes, so the 0x00EB would be broken  
into two bytes and your script would choke on the second 0xEB) : you  
will need to set your encoding to something compatible, like utf-8.

I don’t quite know how to do that properly in IronRuby, but in CRuby 1.9  
you could use “magic comments” in your ruby file and in 1.8 something  
like $KCODE=‘u’ could work. You might also be able to drop back into  
.NET and set the encoding there, but I’m not sure how that affects  
IronRuby assemblies.

I would start with $KCODE = ‘u’ Let me know how that works for you.

Zaki

On Thu, Jan 13, 2011 at 6:33 PM, Albert-Jan Pieter Nijburg  
[removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

Hi Guys,

My boss thought it would be cool to use “” in an sql tablename, many of  
you will want to shoot her now J.

But now I did find something weird, I can’t even print “”.

It says:

tabaco.rb:16:in `puts’: character U+00EB can’t be encoded in US-ASCII  
(Encoding::InvalidByteSequenceError)

```
    from tabaco.rb:16

```

or

when I print the string somewhere else :S when it comes back from a  
method.

System::Text::DecoderFallbackException at /patient/0

Unable to translate bytes [EB] at index 3 from specified code page to  
Unicode.

Or when I don’t mess with it

Encoding::InvalidByteSequenceError at /patient/0

invalid byte sequence EB on UTF-8

All the same problem coming from 3 places.

Is this a fundamental issue or should this be solvable?

If you could point me in the right direction I could try to maybe fix  
it.

Thanks,

Albert-Jan

* * *

Ironruby-core mailing list  
[removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
[http://rubyforge.org/mailman/listinfo/ironruby-core](http://rubyforge.org/mailman/listinfo/ironruby-core)

---

<div class="post-metadata">

**Author:** ![Albert-Jan\_Pieter\_N](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/76d3ee/32.png) [@Albert-Jan\_Pieter\_N](https://www.ruby-forum.com/u/Albert-Jan_Pieter_N)\
**Post date:** [January 13, 2011, 7:15pm UTC](https://www.ruby-forum.com/t/encoding-problem/201718/7 "2011-01-13T19:15:41Z")

</div>

What IronRuby version do you use?

On my machine (github/master):

a.rb (saved as UTF-8 encoded file):

# encoding: UTF-8

a = “ë”  
b = “\u{eb}”

puts a.encoding, b.encoding, a, b, a.inspect, b.inspect

C:\Temp\>rbx a.rb  
UTF-8  
UTF-8  
ë  
ë  
“\u{eb}”  
“\u{eb}”

Which is also what MRI 1.9.2 does.

Tomas

From: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
[[mailto:removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)] On Behalf Of Albert-Jan  
Pieter Nijburg  
Sent: Thursday, January 13, 2011 7:49 AM  
To: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
Subject: Re: [Ironruby-core] Encoding problem

Hey,

I found out that if I put nothing at the top and I do this:

puts “\x89”

it puts “ë”

if I put the #Encoding: UTF-8 at the top this happens. ☺

ëmscorlib:0:in `Throw': Unable to translate bytes [89] at index -1 from specifie d code page to Unicode. (System::Text::DecoderFallbackException) from mscorlib:0:in `Fallback’  
from mscorlib:0:in `InternalFallback' from mscorlib:0:in `GetCharCount’  
from mscorlib:0:in `GetCharCount' from mscorlib:0:in `GetChars’  
from tabaco.rb:2:in `puts’  
from tabaco.rb:2

It does print it but then it dies.

#\<Encoding: UTF-8\>  
puts “\x89”.force\_encoding(“UTF-8”) does the same

#\<Encoding: UTF-8\>  
puts “ë”.force\_encoding(“UTF-8”) does the same as before.  
Also without the #

So I thought I had it with the puts “\x89” and I tried this:

class PatGeg \< ActiveRecord::Base  
set\_table\_name “Pati\x89ntGegevens”  
end

PatGeg.first.Achternaam

and here’s what I got

c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby/gems/1.9.1/g  
ems/activerecord-3.0.0/lib/active\_record/connection\_adapters/abstract\_adapter.rb  
:200:in `log': incompatible character encodings: UTF-8 and ASCII-8BIT (Encoding: :CompatibilityError) from c:/ir/irtest/External.LCA_RESTRICTED/Languages/Ruby/ruby19/lib/ruby /gems/1.9.1/gems/activerecord-sqlserver-adapter-3.0.0/lib/active_record/connecti on_adapters/sqlserver/database_statements.rb:217:in `raw\_select’  
from  
c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby  
/gems/1.9.1/gems/activerecord-sqlserver-adapter-3.0.0/lib/active\_record/connecti  
on\_adapters/sqlserver/database\_statements.rb:178:in `select' from c:/ir/irtest/External.LCA_RESTRICTED/Languages/Ruby/ruby19/lib/ruby /gems/1.9.1/gems/activerecord-3.0.0/lib/active_record/connection_adapters/abstra ct/database_statements.rb:7:in `select\_all’  
from  
c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby  
/gems/1.9.1/gems/activerecord-3.0.0/lib/active\_record/connection\_adapters/abstra  
ct/query\_cache.rb:56:in `select_all' from c:/ir/irtest/External.LCA_RESTRICTED/Languages/Ruby/ruby19/lib/ruby /gems/1.9.1/gems/activerecord-3.0.0/lib/active_record/base.rb:467:in `find\_by\_sq  
l’  
from  
c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby  
/gems/1.9.1/gems/activerecord-3.0.0/lib/active\_record/relation.rb:64:in  
`to_a' from c:/ir/irtest/External.LCA_RESTRICTED/Languages/Ruby/ruby19/lib/ruby /gems/1.9.1/gems/activerecord-3.0.0/lib/active_record/relation/finder_methods.rb :333:in `find\_first’  
from  
c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby  
/gems/1.9.1/gems/activerecord-3.0.0/lib/active\_record/relation/finder\_methods.rb  
:122:in `first' from c:6:in ` **send** ’  
from c:6:in `first’

I’ve tried to do the force\_encoding(“UTF-8”) on this thing to which  
results in something very similar :

c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby/gems/1.9.1/g  
ems/activerecord-sqlserver-adapter-3.0.0/lib/active\_record/connection\_adapters/s  
qlserver/quoting.rb:31:in `=~': invalid byte sequence 89 on UTF-8 (Encoding::Inv alidByteSequenceError) from c:/ir/irtest/External.LCA_RESTRICTED/Languages/Ruby/ruby19/lib/ruby /gems/1.9.1/gems/activerecord-sqlserver-adapter-3.0.0/lib/active_record/connecti on_adapters/sqlserver/quoting.rb:31:in `quote\_table\_name’  
from  
c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby  
/gems/1.9.1/gems/activerecord-3.0.0/lib/active\_record/base.rb:597:in  
`quoted_tab le_name' from c:/ir/irtest/External.LCA_RESTRICTED/Languages/Ruby/ruby19/lib/ruby /gems/1.9.1/gems/activerecord-3.0.0/lib/active_record/relation/query_methods.rb: 234:in `build\_select’  
from  
c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby  
/gems/1.9.1/gems/activerecord-3.0.0/lib/active\_record/relation/query\_methods.rb:  
159:in `build_arel' from c:/ir/irtest/External.LCA_RESTRICTED/Languages/Ruby/ruby19/lib/ruby /gems/1.9.1/gems/activerecord-3.0.0/lib/active_record/relation/query_methods.rb: 110:in `arel’  
from  
c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby  
/gems/1.9.1/gems/activerecord-3.0.0/lib/active\_record/relation.rb:64:in  
`to_a' from c:/ir/irtest/External.LCA_RESTRICTED/Languages/Ruby/ruby19/lib/ruby /gems/1.9.1/gems/activerecord-3.0.0/lib/active_record/relation/finder_methods.rb :333:in `find\_first’  
from  
c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby  
/gems/1.9.1/gems/activerecord-3.0.0/lib/active\_record/relation/finder\_methods.rb  
:122:in `first' from c:6:in ` **send** ’  
from c:6:in `first’  
from tabaco.rb:34

I have a feeling that ironruby and .net are not in sync with the  
encodings

Albert-Jan

Van: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
[[mailto:removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)] Namens Dezso Zoltan  
Verzonden: donderdag 13 januari 2011 16:04  
Aan: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
Onderwerp: Re: [Ironruby-core] Encoding problem

Hi,

> warning: variable $KCODE is no longer effective

This means that you are in 1.9 mode 🙂 In that case there are two things  
you could try:

1. set the encoding at the top of the file in the form of the comment:

# encoding: UTF-8

1. force an encoding on the string(s) in question with the method (if 1)  
fails in IronRuby):  
.force\_encoding(“UTF-8”)

Zaki

On Thu, Jan 13, 2011 at 11:20 PM, Albert-Jan Pieter Nijburg  
\<[removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)[mailto:removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)\> wrote:  
Hey Zaki,

WARNING: YAML.add\_builtin\_type is not implemented  
unknown:0: warning: variable $KCODE is no longer effective  
tabaco.rb:11:in `puts’: character U+00EB can’t be encoded in US-ASCII  
(Encoding:  
:InvalidByteSequenceError)  
from tabaco.rb:11

Too bad.. thanks though. I’ll have a look in the source if I can find  
something.

Annoying Europeans 😛

Albert-Jan

Van:  
[removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)[mailto:removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
[[mailto:ironruby-core-bounces@rubyforge.o](mailto:ironruby-core-bounces@rubyforge.o)[mailto:ironruby-core-bounces@rubyforge.o](mailto:ironruby-core-bounces@rubyforge.o)rg]  
Namens Dezso Zoltan  
Verzonden: donderdag 13 januari 2011 14:52  
Aan: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)[mailto:removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
Onderwerp: Re: [Ironruby-core] Encoding problem

Hi,

I don’t really know the solution to your question, but this might help:  
ë is Unicode U+00EB, which is 0xC3AB in UTF-8 (so we are dealing with  
unicode rather than utf-8, which I assume is because IronRuby uses the  
immutable .NET strings internally with Unicode encoding).

The errors are expected if your default encoding is US-ASCII because it  
does not contain ë (and uses single bytes, so the 0x00EB would be broken  
into two bytes and your script would choke on the second 0xEB) : you  
will need to set your encoding to something compatible, like utf-8.

I don’t quite know how to do that properly in IronRuby, but in CRuby 1.9  
you could use “magic comments” in your ruby file and in 1.8 something  
like $KCODE=‘u’ could work. You might also be able to drop back into  
.NET and set the encoding there, but I’m not sure how that affects  
IronRuby assemblies.

I would start with $KCODE = ‘u’ Let me know how that works for you.

Zaki

On Thu, Jan 13, 2011 at 6:33 PM, Albert-Jan Pieter Nijburg  
\<[removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)[mailto:removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)\> wrote:  
Hi Guys,

My boss thought it would be cool to use “ë” in an sql tablename, many of  
you will want to shoot her now ☺.

But now I did find something weird, I can’t even print “ë”.

It says:

tabaco.rb:16:in `puts’: character U+00EB can’t be encoded in US-ASCII  
(Encoding::InvalidByteSequenceError)  
from tabaco.rb:16

or

when I print the string somewhere else :S when it comes back from a  
method.

System::Text::DecoderFallbackException at /patient/0  
Unable to translate bytes [EB] at index 3 from specified code page to  
Unicode.

Or when I don’t mess with it

Encoding::InvalidByteSequenceError at /patient/0  
invalid byte sequence EB on UTF-8

All the same problem coming from 3 places.

Is this a fundamental issue or should this be solvable?

If you could point me in the right direction I could try to maybe fix  
it.

Thanks,

Albert-Jan

* * *

Ironruby-core mailing list  
[removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)[mailto:removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
[http://rubyforge.org/mailman/listinfo/ironruby-core](http://rubyforge.org/mailman/listinfo/ironruby-core)

---

<div class="post-metadata">

**Author:** ![Albert-Jan\_Pieter\_N](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/76d3ee/32.png) [@Albert-Jan\_Pieter\_N](https://www.ruby-forum.com/u/Albert-Jan_Pieter_N)\
**Post date:** [January 14, 2011, 10:18am UTC](https://www.ruby-forum.com/t/encoding-problem/201718/8 "2011-01-14T10:18:40Z")

</div>

I have a clone from the github repos and it’s at the tip.

It appears that scite does not save its files as UTF-8 by default, I  
assumed it did. Which solves the problem J

Even without the #encoding it works

Thanks

Van: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
[[mailto:removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)] Namens Tomas M.  
Verzonden: donderdag 13 januari 2011 19:13  
Aan: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
Onderwerp: Re: [Ironruby-core] Encoding problem

What IronRuby version do you use?

On my machine (github/master):

a.rb (saved as UTF-8 encoded file):

# encoding: UTF-8

a = “ë”

b = “\u{eb}”

puts a.encoding, b.encoding, a, b, a.inspect, b.inspect

C:\Temp\>rbx a.rb

UTF-8

UTF-8

ë

ë

“\u{eb}”

“\u{eb}”

Which is also what MRI 1.9.2 does.

Tomas

From: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
[[mailto:removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)] On Behalf Of Albert-Jan  
Pieter Nijburg  
Sent: Thursday, January 13, 2011 7:49 AM  
To: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
Subject: Re: [Ironruby-core] Encoding problem

Hey,

I found out that if I put nothing at the top and I do this:

puts “\x89”

it puts “ë”

if I put the #Encoding: UTF-8 at the top this happens. J

ëmscorlib:0:in `Throw’: Unable to translate bytes [89] at index -1 from  
specifie

d code page to Unicode. (System::Text::DecoderFallbackException)

```
    from mscorlib:0:in `Fallback'

    from mscorlib:0:in `InternalFallback'

    from mscorlib:0:in `GetCharCount'

    from mscorlib:0:in `GetCharCount'

    from mscorlib:0:in `GetChars'

    from tabaco.rb:2:in `puts'

    from tabaco.rb:2

```

It does print it but then it dies.

#\<Encoding: UTF-8\>

puts “\x89”.force\_encoding(“UTF-8”) does the same

#\<Encoding: UTF-8\>

puts “ë”.force\_encoding(“UTF-8”) does the same as before.

Also without the #

So I thought I had it with the puts “\x89” and I tried this:

class PatGeg \< ActiveRecord::Base

```
  set_table_name "Pati\x89ntGegevens"

```

end

PatGeg.first.Achternaam

and here’s what I got

c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby/gems/1.9.1/g

ems/activerecord-3.0.0/lib/active\_record/connection\_adapters/abstract\_adapter.rb

:200:in `log’: incompatible character encodings: UTF-8 and ASCII-8BIT  
(Encoding:

:CompatibilityError)

```
    from 

```

c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby

/gems/1.9.1/gems/activerecord-sqlserver-adapter-3.0.0/lib/active\_record/connecti

on\_adapters/sqlserver/database\_statements.rb:217:in `raw\_select’

```
    from 

```

c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby

/gems/1.9.1/gems/activerecord-sqlserver-adapter-3.0.0/lib/active\_record/connecti

on\_adapters/sqlserver/database\_statements.rb:178:in `select’

```
    from 

```

c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby

/gems/1.9.1/gems/activerecord-3.0.0/lib/active\_record/connection\_adapters/abstra

ct/database\_statements.rb:7:in `select\_all’

```
    from 

```

c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby

/gems/1.9.1/gems/activerecord-3.0.0/lib/active\_record/connection\_adapters/abstra

ct/query\_cache.rb:56:in `select\_all’

```
    from 

```

c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby

/gems/1.9.1/gems/activerecord-3.0.0/lib/active\_record/base.rb:467:in  
`find\_by\_sq

l’

```
    from 

```

c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby

/gems/1.9.1/gems/activerecord-3.0.0/lib/active\_record/relation.rb:64:in  
`to\_a’

```
    from 

```

c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby

/gems/1.9.1/gems/activerecord-3.0.0/lib/active\_record/relation/finder\_methods.rb

:333:in `find\_first’

```
    from 

```

c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby

/gems/1.9.1/gems/activerecord-3.0.0/lib/active\_record/relation/finder\_methods.rb

:122:in `first’

```
    from c:6:in ` __send__'

    from c:6:in `first'

```

I’ve tried to do the force\_encoding(“UTF-8”) on this thing to which  
results in something very similar :

c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby/gems/1.9.1/g

ems/activerecord-sqlserver-adapter-3.0.0/lib/active\_record/connection\_adapters/s

qlserver/quoting.rb:31:in `=~': invalid byte sequence 89 on UTF-8  
(Encoding::Inv

alidByteSequenceError)

```
    from 

```

c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby

/gems/1.9.1/gems/activerecord-sqlserver-adapter-3.0.0/lib/active\_record/connecti

on\_adapters/sqlserver/quoting.rb:31:in `quote\_table\_name’

```
    from 

```

c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby

/gems/1.9.1/gems/activerecord-3.0.0/lib/active\_record/base.rb:597:in  
`quoted\_tab

le\_name’

```
    from 

```

c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby

/gems/1.9.1/gems/activerecord-3.0.0/lib/active\_record/relation/query\_methods.rb:

234:in `build\_select’

```
    from 

```

c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby

/gems/1.9.1/gems/activerecord-3.0.0/lib/active\_record/relation/query\_methods.rb:

159:in `build\_arel’

```
    from 

```

c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby

/gems/1.9.1/gems/activerecord-3.0.0/lib/active\_record/relation/query\_methods.rb:

110:in `arel’

```
    from 

```

c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby

/gems/1.9.1/gems/activerecord-3.0.0/lib/active\_record/relation.rb:64:in  
`to\_a’

```
    from 

```

c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby

/gems/1.9.1/gems/activerecord-3.0.0/lib/active\_record/relation/finder\_methods.rb

:333:in `find\_first’

```
    from 

```

c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby

/gems/1.9.1/gems/activerecord-3.0.0/lib/active\_record/relation/finder\_methods.rb

:122:in `first’

```
    from c:6:in ` __send__'

    from c:6:in `first'

    from tabaco.rb:34

```

I have a feeling that ironruby and .net are not in sync with the  
encodings

Albert-Jan

Van: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
[[mailto:removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)] Namens Dezso Zoltan  
Verzonden: donderdag 13 januari 2011 16:04  
Aan: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
Onderwerp: Re: [Ironruby-core] Encoding problem

Hi,

> warning: variable $KCODE is no longer effective

This means that you are in 1.9 mode 🙂 In that case there are two things  
you could try:

1. set the encoding at the top of the file in the form of the comment:

# encoding: UTF-8

1. force an encoding on the string(s) in question with the method (if 1)  
fails in IronRuby):

.force\_encoding(“UTF-8”)

Zaki

On Thu, Jan 13, 2011 at 11:20 PM, Albert-Jan Pieter Nijburg  
[removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

Hey Zaki,

WARNING: YAML.add\_builtin\_type is not implemented

unknown:0: warning: variable $KCODE is no longer effective

tabaco.rb:11:in `puts’: character U+00EB can’t be encoded in US-ASCII  
(Encoding:

:InvalidByteSequenceError)

```
    from tabaco.rb:11

```

Too bad.. thanks though. I’ll have a look in the source if I can find  
something.

Annoying Europeans 😛

Albert-Jan

Van: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
[[mailto:removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)] Namens Dezso Zoltan  
Verzonden: donderdag 13 januari 2011 14:52  
Aan: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
Onderwerp: Re: [Ironruby-core] Encoding problem

Hi,

I don’t really know the solution to your question, but this might help:

ë is Unicode U+00EB, which is 0xC3AB in UTF-8 (so we are dealing with  
unicode rather than utf-8, which I assume is because IronRuby uses the  
immutable .NET strings internally with Unicode encoding).

The errors are expected if your default encoding is US-ASCII because it  
does not contain ë (and uses single bytes, so the 0x00EB would be broken  
into two bytes and your script would choke on the second 0xEB) : you  
will need to set your encoding to something compatible, like utf-8.

I don’t quite know how to do that properly in IronRuby, but in CRuby 1.9  
you could use “magic comments” in your ruby file and in 1.8 something  
like $KCODE=‘u’ could work. You might also be able to drop back into  
.NET and set the encoding there, but I’m not sure how that affects  
IronRuby assemblies.

I would start with $KCODE = ‘u’ Let me know how that works for you.

Zaki

On Thu, Jan 13, 2011 at 6:33 PM, Albert-Jan Pieter Nijburg  
[removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

Hi Guys,

My boss thought it would be cool to use “ë” in an sql tablename, many of  
you will want to shoot her now J.

But now I did find something weird, I can’t even print “ë”.

It says:

tabaco.rb:16:in `puts’: character U+00EB can’t be encoded in US-ASCII  
(Encoding::InvalidByteSequenceError)

```
    from tabaco.rb:16

```

or

when I print the string somewhere else :S when it comes back from a  
method.

System::Text::DecoderFallbackException at /patient/0

Unable to translate bytes [EB] at index 3 from specified code page to  
Unicode.

Or when I don’t mess with it

Encoding::InvalidByteSequenceError at /patient/0

invalid byte sequence EB on UTF-8

All the same problem coming from 3 places.

Is this a fundamental issue or should this be solvable?

If you could point me in the right direction I could try to maybe fix  
it.

Thanks,

Albert-Jan

* * *

Ironruby-core mailing list  
[removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
[http://rubyforge.org/mailman/listinfo/ironruby-core](http://rubyforge.org/mailman/listinfo/ironruby-core)

---

<div class="post-metadata">

**Author:** ![Albert-Jan\_Pieter\_N](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/76d3ee/32.png) [@Albert-Jan\_Pieter\_N](https://www.ruby-forum.com/u/Albert-Jan_Pieter_N)\
**Post date:** [January 14, 2011, 7:24pm UTC](https://www.ruby-forum.com/t/encoding-problem/201718/9 "2011-01-14T19:24:37Z")

</div>

Most editors insert BOM character at the beginning of the file. This is  
a byte sequence that allows readers to identify Unicode encoding and is  
not usually visible in editors.

Tomas

From: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
[[mailto:removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)] On Behalf Of Albert-Jan  
Pieter Nijburg  
Sent: Friday, January 14, 2011 12:49 AM  
To: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
Subject: Re: [Ironruby-core] Encoding problem

I have a clone from the github repos and it’s at the tip.

It appears that scite does not save its files as UTF-8 by default, I  
assumed it did. Which solves the problem ☺

Even without the #encoding it works

Thanks

Van: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
[[mailto:removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)] Namens Tomas M.  
Verzonden: donderdag 13 januari 2011 19:13  
Aan: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
Onderwerp: Re: [Ironruby-core] Encoding problem

What IronRuby version do you use?

On my machine (github/master):

a.rb (saved as UTF-8 encoded file):

# encoding: UTF-8

a = “ë”  
b = “\u{eb}”

puts a.encoding, b.encoding, a, b, a.inspect, b.inspect

C:\Temp\>rbx a.rb  
UTF-8  
UTF-8  
ë  
ë  
“\u{eb}”  
“\u{eb}”

Which is also what MRI 1.9.2 does.

Tomas

From: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
[[mailto:removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)] On Behalf Of Albert-Jan  
Pieter Nijburg  
Sent: Thursday, January 13, 2011 7:49 AM  
To: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
Subject: Re: [Ironruby-core] Encoding problem

Hey,

I found out that if I put nothing at the top and I do this:

puts “\x89”

it puts “ë”

if I put the #Encoding: UTF-8 at the top this happens. ☺

ëmscorlib:0:in `Throw': Unable to translate bytes [89] at index -1 from specifie d code page to Unicode. (System::Text::DecoderFallbackException) from mscorlib:0:in `Fallback’  
from mscorlib:0:in `InternalFallback' from mscorlib:0:in `GetCharCount’  
from mscorlib:0:in `GetCharCount' from mscorlib:0:in `GetChars’  
from tabaco.rb:2:in `puts’  
from tabaco.rb:2

It does print it but then it dies.

#\<Encoding: UTF-8\>  
puts “\x89”.force\_encoding(“UTF-8”) does the same

#\<Encoding: UTF-8\>  
puts “ë”.force\_encoding(“UTF-8”) does the same as before.  
Also without the #

So I thought I had it with the puts “\x89” and I tried this:

class PatGeg \< ActiveRecord::Base  
set\_table\_name “Pati\x89ntGegevens”  
end

PatGeg.first.Achternaam

and here’s what I got

c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby/gems/1.9.1/g  
ems/activerecord-3.0.0/lib/active\_record/connection\_adapters/abstract\_adapter.rb  
:200:in `log': incompatible character encodings: UTF-8 and ASCII-8BIT (Encoding: :CompatibilityError) from c:/ir/irtest/External.LCA_RESTRICTED/Languages/Ruby/ruby19/lib/ruby /gems/1.9.1/gems/activerecord-sqlserver-adapter-3.0.0/lib/active_record/connecti on_adapters/sqlserver/database_statements.rb:217:in `raw\_select’  
from  
c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby  
/gems/1.9.1/gems/activerecord-sqlserver-adapter-3.0.0/lib/active\_record/connecti  
on\_adapters/sqlserver/database\_statements.rb:178:in `select' from c:/ir/irtest/External.LCA_RESTRICTED/Languages/Ruby/ruby19/lib/ruby /gems/1.9.1/gems/activerecord-3.0.0/lib/active_record/connection_adapters/abstra ct/database_statements.rb:7:in `select\_all’  
from  
c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby  
/gems/1.9.1/gems/activerecord-3.0.0/lib/active\_record/connection\_adapters/abstra  
ct/query\_cache.rb:56:in `select_all' from c:/ir/irtest/External.LCA_RESTRICTED/Languages/Ruby/ruby19/lib/ruby /gems/1.9.1/gems/activerecord-3.0.0/lib/active_record/base.rb:467:in `find\_by\_sq  
l’  
from  
c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby  
/gems/1.9.1/gems/activerecord-3.0.0/lib/active\_record/relation.rb:64:in  
`to_a' from c:/ir/irtest/External.LCA_RESTRICTED/Languages/Ruby/ruby19/lib/ruby /gems/1.9.1/gems/activerecord-3.0.0/lib/active_record/relation/finder_methods.rb :333:in `find\_first’  
from  
c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby  
/gems/1.9.1/gems/activerecord-3.0.0/lib/active\_record/relation/finder\_methods.rb  
:122:in `first' from c:6:in ` **send** ’  
from c:6:in `first’

I’ve tried to do the force\_encoding(“UTF-8”) on this thing to which  
results in something very similar :

c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby/gems/1.9.1/g  
ems/activerecord-sqlserver-adapter-3.0.0/lib/active\_record/connection\_adapters/s  
qlserver/quoting.rb:31:in `=~': invalid byte sequence 89 on UTF-8 (Encoding::Inv alidByteSequenceError) from c:/ir/irtest/External.LCA_RESTRICTED/Languages/Ruby/ruby19/lib/ruby /gems/1.9.1/gems/activerecord-sqlserver-adapter-3.0.0/lib/active_record/connecti on_adapters/sqlserver/quoting.rb:31:in `quote\_table\_name’  
from  
c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby  
/gems/1.9.1/gems/activerecord-3.0.0/lib/active\_record/base.rb:597:in  
`quoted_tab le_name' from c:/ir/irtest/External.LCA_RESTRICTED/Languages/Ruby/ruby19/lib/ruby /gems/1.9.1/gems/activerecord-3.0.0/lib/active_record/relation/query_methods.rb: 234:in `build\_select’  
from  
c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby  
/gems/1.9.1/gems/activerecord-3.0.0/lib/active\_record/relation/query\_methods.rb:  
159:in `build_arel' from c:/ir/irtest/External.LCA_RESTRICTED/Languages/Ruby/ruby19/lib/ruby /gems/1.9.1/gems/activerecord-3.0.0/lib/active_record/relation/query_methods.rb: 110:in `arel’  
from  
c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby  
/gems/1.9.1/gems/activerecord-3.0.0/lib/active\_record/relation.rb:64:in  
`to_a' from c:/ir/irtest/External.LCA_RESTRICTED/Languages/Ruby/ruby19/lib/ruby /gems/1.9.1/gems/activerecord-3.0.0/lib/active_record/relation/finder_methods.rb :333:in `find\_first’  
from  
c:/ir/irtest/External.LCA\_RESTRICTED/Languages/Ruby/ruby19/lib/ruby  
/gems/1.9.1/gems/activerecord-3.0.0/lib/active\_record/relation/finder\_methods.rb  
:122:in `first' from c:6:in ` **send** ’  
from c:6:in `first’  
from tabaco.rb:34

I have a feeling that ironruby and .net are not in sync with the  
encodings

Albert-Jan

Van: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
[[mailto:removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)] Namens Dezso Zoltan  
Verzonden: donderdag 13 januari 2011 16:04  
Aan: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
Onderwerp: Re: [Ironruby-core] Encoding problem

Hi,

> warning: variable $KCODE is no longer effective

This means that you are in 1.9 mode 🙂 In that case there are two things  
you could try:

1. set the encoding at the top of the file in the form of the comment:

# encoding: UTF-8

1. force an encoding on the string(s) in question with the method (if 1)  
fails in IronRuby):  
.force\_encoding(“UTF-8”)

Zaki

On Thu, Jan 13, 2011 at 11:20 PM, Albert-Jan Pieter Nijburg  
\<[removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)[mailto:removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)\> wrote:  
Hey Zaki,

WARNING: YAML.add\_builtin\_type is not implemented  
unknown:0: warning: variable $KCODE is no longer effective  
tabaco.rb:11:in `puts’: character U+00EB can’t be encoded in US-ASCII  
(Encoding:  
:InvalidByteSequenceError)  
from tabaco.rb:11

Too bad.. thanks though. I’ll have a look in the source if I can find  
something.

Annoying Europeans 😛

Albert-Jan

Van:  
[removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)[mailto:removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
[[mailto:ironruby-core-bounces@rubyforge.o](mailto:ironruby-core-bounces@rubyforge.o)[mailto:ironruby-core-bounces@rubyforge.o](mailto:ironruby-core-bounces@rubyforge.o)rg]  
Namens Dezso Zoltan  
Verzonden: donderdag 13 januari 2011 14:52  
Aan: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)[mailto:removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
Onderwerp: Re: [Ironruby-core] Encoding problem

Hi,

I don’t really know the solution to your question, but this might help:  
ë is Unicode U+00EB, which is 0xC3AB in UTF-8 (so we are dealing with  
unicode rather than utf-8, which I assume is because IronRuby uses the  
immutable .NET strings internally with Unicode encoding).

The errors are expected if your default encoding is US-ASCII because it  
does not contain ë (and uses single bytes, so the 0x00EB would be broken  
into two bytes and your script would choke on the second 0xEB) : you  
will need to set your encoding to something compatible, like utf-8.

I don’t quite know how to do that properly in IronRuby, but in CRuby 1.9  
you could use “magic comments” in your ruby file and in 1.8 something  
like $KCODE=‘u’ could work. You might also be able to drop back into  
.NET and set the encoding there, but I’m not sure how that affects  
IronRuby assemblies.

I would start with $KCODE = ‘u’ Let me know how that works for you.

Zaki

On Thu, Jan 13, 2011 at 6:33 PM, Albert-Jan Pieter Nijburg  
\<[removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)[mailto:removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)\> wrote:  
Hi Guys,

My boss thought it would be cool to use “ë” in an sql tablename, many of  
you will want to shoot her now ☺.

But now I did find something weird, I can’t even print “ë”.

It says:

tabaco.rb:16:in `puts’: character U+00EB can’t be encoded in US-ASCII  
(Encoding::InvalidByteSequenceError)  
from tabaco.rb:16

or

when I print the string somewhere else :S when it comes back from a  
method.

System::Text::DecoderFallbackException at /patient/0  
Unable to translate bytes [EB] at index 3 from specified code page to  
Unicode.

Or when I don’t mess with it

Encoding::InvalidByteSequenceError at /patient/0  
invalid byte sequence EB on UTF-8

All the same problem coming from 3 places.

Is this a fundamental issue or should this be solvable?

If you could point me in the right direction I could try to maybe fix  
it.

Thanks,

Albert-Jan

* * *

Ironruby-core mailing list  
[removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)[mailto:removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
[http://rubyforge.org/mailman/listinfo/ironruby-core](http://rubyforge.org/mailman/listinfo/ironruby-core)