Problem with encoding (Umlaut: ü, ä...)

Hi there from Germany,

I just installed and set up ferret and act_as_ferret for rails. All of
them at the most recent version. The development environment is running
fine with Mac OS X. But I got problems with the productive environment
(debian).

In the (mysql-)database are few records stored, containig german umlauts
(ü for example). Running a query for “köln” returns the correct record
in dev-environment but NOTHING at the debian system. But the logs are
looking good. It seems the word was correctly submited:

Processing SearchController#result (for 127.0.0.1 at 2007-03-31
23:45:47) [POST]
Session ID: 55f4544e0b28e991a1460b05dc09744c
Parameters: {“commit”=>“suchen”, “action”=>“result”,
“controller”=>“search”, “query”=>“köln”}

I read a few things here in the forum and elsewhere, but not find any
solution. Would you please give me some pointers? I really get not
through these encoding, locales, collation and stuff.

What do I have to configure how?

It would be really nice if you could help.

On Mar 31, 2007, at 11:56 PM, Rainer Kern wrote:

looking good. It seems the word was correctly submited:
Your system locale should be set to UTF-8. Use the ‘locale’ command
to view the current settings and change the LANG and LC_ALL
environment variables if necessary. (In your case they should
probably be set to “de_DE.UTF-8”)

MySQL should be configured to use UTF-8 as well. You can either
specify the character set for each individual table (via the CREATE
TABLE command) or globally in my.cnf:

character-set-server = utf8
default-character-set = utf8

Finally, for Rails, add the following lines to environment.rb:

$KCODE = ‘u’
require ‘jcode’

I recommend to use UTF-8 throughout the whole stack (OS, MySQL,
Rails). That way you’ll get rid of your encoding problems once and
for all.

Cheers,
Andy

Hey Andy,

Andreas K. wrote:

On Mar 31, 2007, at 11:56 PM, Rainer Kern wrote:

looking good. It seems the word was correctly submited:
[…] Your system locale should be set to UTF-8. Use the ‘locale’ command
I recommend to use UTF-8 throughout the whole stack (OS, MySQL,
Rails). That way you’ll get rid of your encoding problems once and
for all.

BIG thanks for that. It was of great help. Now, running the rails web
server Webrick, works with UTF-8. I’ve set the database tables to UTF-8,
locale to de_DE.UTF-8@euro and added the suggested variables in
environment.rb.

The one problem left is, that it still does not work in
production-environment running with apache2 und fcgi. This is rails
specific. But is there anything I forgot to set in the apache-config? I
wonder why it is runnning with webrick but not with apache.

Would you please comment on that? That would be nice.

rainer

Rainer Kern wrote:
… forgot to mention that I’ve set the following in my apache2 virtual
host:

AddDefaultCharset utf-8

This does not help!

Any hint?

Got it!

Had to set

ENV[‘LANG’] = ‘de_DE.UTF-8@euro’

in config/environment.rb even though I had locales set right.

Thanks.

It might be helpful for someone to read the conclusion about this topic
at http://www.dockblog.de/

On Sun, Apr 01, 2007 at 05:16:42PM +0200, Rainer Kern wrote:

It might be helpful for someone to read the conclusion about this topic
at http://www.dockblog.de/

cool, I added a link to this to the Wiki.

Jens


Jens Krämer
webit! Gesellschaft für neue Medien mbH
Schnorrstraße 76 | 01069 Dresden
Telefon +49 351 46766-0 | Telefax +49 351 46766-66
[email protected] | www.webit.de

Amtsgericht Dresden | HRB 15422
GF Sven Haubold, Hagen Malessa

Rainer Kern wrote:

Rainer Kern wrote:
… forgot to mention that I’ve set the following in my apache2 virtual
host:

AddDefaultCharset utf-8

This does not help!

Any hint?

… I’m still on this problem. Found out that ferret_index.log comes up
with

query: txt:BAD_DATA owner_name:BAD_DATA seen_where:BAD_DATA

when submitting “Köln” in the search form.

Google gives me nothing back when searching for “ferret query
bad_data”. What is so bad with my data?

Jens K. wrote:

On Sun, Apr 01, 2007 at 05:16:42PM +0200, Rainer Kern wrote:

It might be helpful for someone to read the conclusion about this topic
at http://www.dockblog.de/

cool, I added a link to this to the Wiki.

Thanks. Hopfully someone will find it helpful too.