Forum: Ruby on Rails mysql encoding with rails 2.3.2 and ruby 1.9.1

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
ruby.freeman (Guest)
on 2009-03-21 16:53
(Received via mailing list)
when I run console with ruby 1.9.1 and rails 2.3.2, and trying to do
something like

User.first.name.encoding

I'm getting #<Encoding:ASCII-8BIT>, though I've set "encoding: utf8"
in database.yml

any suggestions?
Jeremy K. (Guest)
on 2009-03-22 05:19
(Received via mailing list)
On Sat, Mar 21, 2009 at 7:52 AM, ruby.freeman 
<removed_email_address@domain.invalid>
wrote:
>
> when I run console with ruby 1.9.1 and rails 2.3.2, and trying to do
> something like
>
> User.first.name.encoding
>
> I'm getting #<Encoding:ASCII-8BIT>, though I've set "encoding: utf8"
> in database.yml
>
> any suggestions?

The mysql driver is not encoding-aware yet.

jeremy
Stefan Â. (Guest)
on 2009-03-23 19:16
Jeremy K. wrote:
> On Sat, Mar 21, 2009 at 7:52 AM, ruby.freeman <removed_email_address@domain.invalid>
> wrote:
>>
>> when I run console with ruby 1.9.1 and rails 2.3.2, and trying to do
>> something like
>>
>> User.first.name.encoding
>>
>> I'm getting #<Encoding:ASCII-8BIT>, though I've set "encoding: utf8"
>> in database.yml
>>
>> any suggestions?
>
> The mysql driver is not encoding-aware yet.
>
> jeremy

Same problem here with mysql and sqlite3.

When will the mysql and sqlite3 drivers be encoding-aware?

Is there a workaround?

I have problems with this because I'm from germany and need utf8 for the
german umlauts.

If i save the templates as ANSI, the outputs from database with umlauts
are working, but when i use umlauts directly in the templates i get this
error:

invalid byte sequence for encoding "ASCII-8BIT"

If i save the templates as UTF8, the umlauts in the templates are
working, but the outputs from the database with umlauts cause this
error:

invalid byte sequence for encoding "UTF-8"


I'm using Ruby 1.9.1, Rails 2.3.2, mysql/ruby 2.8.1, sqlite3/ruby 1.2.4

slang17
Conrad T. (Guest)
on 2009-03-23 20:55
(Received via mailing list)
On Mon, Mar 23, 2009 at 10:16 AM, Stefan La <
removed_email_address@domain.invalid> wrote:

> >> I'm getting #<Encoding:ASCII-8BIT>, though I've set "encoding: utf8"
> When will the mysql and sqlite3 drivers be encoding-aware?
> invalid byte sequence for encoding "ASCII-8BIT"
> slang17
Your question should also be posted to driver maintainers and possible
the
MySQL and SQLite projects.  Furthermore, all these projects are
open-source.
Thus, please feel free to contribute/add these features if you need
them.

Good luck,

-Conrad
ruby.freeman (Guest)
on 2009-04-04 20:12
(Received via mailing list)
> Your question should also be posted to driver maintainers and possible the
> MySQL and SQLite projects.  Furthermore, all these projects are open-source.
> Thus, please feel free to contribute/add these features if you need them.

It's problem of rails, not mysql library. ActiveRecord should be
encoding-aware now
Stefan Â. (Guest)
on 2009-04-06 13:57
ruby.freeman wrote:

> It's problem of rails, not mysql library. ActiveRecord should be
> encoding-aware now

Where did you get that information?
Is there any workaround or patch for that problem with ActiveRecord?
ruby.freeman (Guest)
on 2009-04-06 14:43
(Received via mailing list)
> Where did you get that information?
> Is there any workaround or patch for that problem with ActiveRecord?

See, when I do some requests from rails console, I see that result is
in utf-8 encoding, so mysql works correctly. But ActiveRecord sets
encoding for all attributes as ASCII-8BIT. For now we can reencode all
results in our views with "force_encoding" methods untill AcriveRecord
is encoding-aware
Conrad T. (Guest)
on 2009-04-06 16:51
(Received via mailing list)
On Mon, Apr 6, 2009 at 3:42 AM, ruby.freeman 
<removed_email_address@domain.invalid>
wrote:

>
> > Where did you get that information?
> > Is there any workaround or patch for that problem with ActiveRecord?
>
> See, when I do some requests from rails console, I see that result is
> in utf-8 encoding, so mysql works correctly. But ActiveRecord sets
> encoding for all attributes as ASCII-8BIT. For now we can reencode all
> results in our views with "force_encoding" methods untill AcriveRecord
> is encoding-aware


Which requests do you perform in the rails console?

-Conrad
ruby.freeman (Guest)
on 2009-04-06 17:01
(Received via mailing list)
> Which requests do you perform in the rails console?

any find request, like in first message
Hector G. (Guest)
on 2009-04-11 12:33
(Received via mailing list)
I made a bug in lighhouse to track this problems, I have some patches
for mysql and for the problem with the views in a related ticket
https://rails.lighthouseapp.com/projects/8994/tick...
Rick (Guest)
on 2009-04-11 22:08
(Received via mailing list)
Postgresql v6.3.8.1 works with Ruby 1.9.1p0 and Rails 2.3.2 - my guess
is the problem is with either the driver or the database itself.

Can you tell what the actual encoding is of the database?  How about
setting the default? MySQL refers to this as the CHARACTER SET.  For
example, to set utf8 as the encoding for a table named
"test_development" you could use:

ALTER TABLE `test_development`.`pages` CHARACTER SET utf8;

Noodle around with the MySQL user's manual and I'm sure you'll find
the way to make utf8 the default for all new tables.

For reference, I've got:

mysql  Ver 14.14 Distrib 5.1.31, for apple-darwin9.5.0 (powerpc) using
readline 5.1

and

gem list mysql == mysql (2.8.1)
ruby.freeman (Guest)
on 2009-04-11 22:29
(Received via mailing list)
> Postgresql v6.3.8.1 works with Ruby 1.9.1p0 and Rails 2.3.2 - my guess
> is the problem is with either the driver or the database itself.

Problem is not with database. As I said above, when I do something
like User.first.name.encoding I'm getting #<Encoding:ASCII-8BIT>.
If I do this one - User.first.name.force_encoding("utf-8") - I'm
getting normal utf-8 string. The problem is that ActiveRecord gets all
data from database as ASCII.
Rick (Guest)
on 2009-04-11 22:45
(Received via mailing list)
right - somewhat slow on the uptake
Hector G. (Guest)
on 2009-04-12 00:52
(Received via mailing list)
ruby.freeman have you checked my fork of mysql-ruby? and the ticket
in lighthouse?
I think this will resolve your problem.
Or have you checked it out and still got problems?
Conrad T. (Guest)
on 2009-04-12 02:32
(Received via mailing list)
On Sat, Apr 11, 2009 at 1:51 PM, Hector G. 
<removed_email_address@domain.invalid>
wrote:

> > Problem is not with database. As I said above, when I do something
> > like User.first.name.encoding I'm getting #<Encoding:ASCII-8BIT>.
> > If I do this one - User.first.name.force_encoding("utf-8") - I'm
> > getting normal utf-8 string. The problem is that ActiveRecord gets all
> > data from database as ASCII.
>

Is your update forked back into the main branch for the project?

-Conrad
Hector G. (Guest)
on 2009-04-12 04:07
(Received via mailing list)
I have not contacted upstream (tmtm) but he is working on a pure ruby
adapter called ruby-mysql (is encoding aware, etc) it has a compat
layer but is
not functional now.
I have made a gem of my fork hectoregm-mysql-ruby that you
use to make tests, but I will try to get this changes merge upstream.
This topic is locked and can not be replied to.