[ruby talk] Localization woes

Rails
1

People,

I’m trying to use the Localization Generator written by Joe Hosteny:
http://rubyforge.org/projects/localization/

The idea behind it is simple.

For example, for Spanish, create a file named sp.yaml and fill it with
key value pairs:

hello: hola
here: aquí
Spanish: Español

Then when I need to translate a key into a value I do something like
this:

@msg = l(:Spanish)

So everything seems to work okay if I put plain characters in sp.yaml.

Webrick, however, spazzes out if I put non-English characters in
sp.yaml:

script/server
=> Booting WEBrick…
./script/…/config/…/lib/localization.rb:62:in iconv': "叩 aqué±? como" (Iconv::IllegalSequence) from ./script/../config/../lib/localization.rb:62:in load_localized_strings’
from ./script/…/config/…/lib/localization.rb:61:in each' from ./script/../config/../lib/localization.rb:61:in load_localized_strings’
from ./script/…/config/…/lib/localization.rb:60:in open' from ./script/../config/../lib/localization.rb:60:in load_localized_strings’
from ./script/…/config/…/lib/localization.rb:52:in each' from ./script/../config/../lib/localization.rb:52:in load_localized_strings’
from ./script/…/config/…/config/environment.rb:59
… 8 levels…
from
/opt/local/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:21:in
require__' from /opt/local/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:21:in require’
from
/opt/local/lib/ruby/gems/1.8/gems/activesupport-1.2.3/lib/active_support/dependencies.rb:214:in
`require’
from script/server:3
zmac11:~/pt/TThier/Languages/ruby/j413 oracle$

Notice the msg:
./script/…/config/…/lib/localization.rb:62:in `iconv’: “叩 aqué±? como”
(Iconv::IllegalSequence)

It was caused by this line in my sp.yaml:
you_are_here_as: usted está aquí como

Obviously it had problems with
está
and
aquí

I used emacs to create sp.yaml and emacs asked me which character set
encoding I want to use.

I told it utf-8 and emacs saved sp.yaml just fine.

I pulled the actual characters from google and used my mouse to copy
them into emacs:

I checked the encoding my browser was using on the google page and it
told me it was using: UTF8
which I’ll assume is the same as the utf-8 I’m using in emacs.

Do any of you have some sample localization yaml files which you’d be
willing to share with me?

How about some general troubleshooting ideas?

Thanks,
-Dan

2

Dan B. wrote:

People,

I’m trying to use the Localization Generator written by Joe Hosteny:
http://rubyforge.org/projects/localization/

You will get better, faster, help for Rails at

http://lists.rubyonrails.org/mailman/listinfo/rails

–

http://www.ruby-doc.org - Ruby Help & Documentation
Ruby Code & Style - Ruby Code & Style: Writers wanted
http://www.rubystuff.com - The Ruby Store for Ruby Stuff
http://www.jamesbritt.com - Playing with Better Toys
http://www.30secondrule.com - Building Better Tools

3

On 12/5/05, James B. [email protected] wrote:

Dan B. wrote:

People,

I’m trying to use the Localization Generator written by Joe Hosteny:
http://rubyforge.org/projects/localization/

You will get better, faster, help for Rails at

This seems to be a yaml / webrick issue though, and not a rails one.

4

Dan B. escribió:

hello: hola

from ./script/…/config/…/lib/localization.rb:52:in `each’
zmac11:~/pt/TThier/Languages/ruby/j413 oracle$
and

Thanks,
-Dan

Rails mailing list
[email protected]
http://lists.rubyonrails.org/mailman/listinfo/rails

I think that yaml is not the best file format for storing translations,
as I ran with a similar problem when trying to store some translations
with HTML embedded…
Maybe the’d better look for another file format for that.

5

Joe Van D. wrote:

You will get better, faster, help for Rails at

This seems to be a yaml / webrick issue though, and not a rails one.

My apolgies.

James

6

Can I strongly reccommend the Globalize plugin instead?

http://globalize.diluvia.net/wiki

7

I cant get it to work.

The wiki suggests this syntax:
script/plugin install
svn://www.diluvia.net/rails-plugins/globalize/trunk

I see this:

/opt/local/lib/ruby/1.8/open-uri.rb:87:in `initialize’: No such file
or directory - svn://www.diluvia.net/rails-plugins/globalize/trunk
(Errno::ENOENT)

Anyone out there get this globalize plugin to work???

-Dan

8

What version of Ruby and Rails are you on? Do you have subversion
installed? With the most current stuff and subversion I can verify
that it works for me:

Mac

cd /tmp
rails test
cd test
script/plugin install
svn://www.diluvia.net/rails-plugins/globalize/trunk

My environment:
ruby -v
ruby 1.8.3 (2005-09-21) [powerpc-darwin8.3.0]
gem list --local

*** LOCAL GEMS ***

actionmailer (1.1.4, 1.1.3)
Service layer for easy email delivery and testing.

actionpack (1.11.1, 1.11.0)
Web-flow and rendering framework putting the VC in MVC.

actionwebservice (0.9.4, 0.9.3)
Web service support for Action Pack.

activerecord (1.13.1, 1.13.0)
Implements the ActiveRecord pattern for ORM.

activesupport (1.2.4, 1.2.3)
Support and utility classes used by the Rails framework.

fastercsv (0.1.4, 0.1.3)
FasterCSV is CSV, but faster, smaller, and cleaner.

fcgi (0.8.6.1)
FastCGI ruby binding.

fixrbconfig (1.2)
Fixes the rbconfig.rb that ships with Mac OS X 10.4 (Tiger), which
makes it impossible to compile C extensions

needle (1.2.1)
Needle is a Dependency Injection/Inversion of Control container for
Ruby. It supports both type-2 (setter) and type-3 (constructor)
injection. It takes advantage of the dynamic nature of Ruby to
provide a rich and flexible approach to injecting dependencies.

net-ssh (1.0.3)
Net::SSH is a pure-Ruby implementation of the SSH2 client protocol.

rails (0.14.4)
Web-application framework with template engine, control-flow layer,
and ORM.

rake (0.6.2)
Ruby based make-like utility.

rmagick (1.9.3)
RMagick is an interface between the Ruby programming language and
the ImageMagick and GraphicsMagick image processing libraries.

rubyzip (0.5.12)
rubyzip is a ruby module for reading and writing zip files

sources (0.0.1)
This package provides download sources for remote gem installation

sqlite3-ruby (1.1.0)
SQLite3/Ruby is a module to allow Ruby scripts to interface with a
SQLite3 database.

switchtower (0.9.0)
SwitchTower is a framework and utility for executing commands in
parallel on multiple remote machines, via SSH. The primary goal
is to simplify and automate the deployment of web applications.

unicode (0.1)
Unicode normalization library.

9

Hi Dan.

It also works for me fine. I am on WinXP, Ruby 1.8.2-15, RoR RC5, svn
1.2.3(r15833).

Also as workaround you could checkout sources by hands. Just do
cd vendor/plugins
svn co svn://www.diluvia.net/rails-plugins/globalize/trunk globalize

10

231:~/CD/railsdemos/globize oracle$ ruby -v
ruby 1.8.2 (2004-12-25) [powerpc-darwin7.9.0]
231:~/CD/railsdemos/globize oracle$ gem list --local

*** LOCAL GEMS ***

actionmailer (1.1.3.3225, 1.1.3)
Service layer for easy email delivery and testing.

actionpack (1.11.0.3225, 1.11.0)
Web-flow and rendering framework putting the VC in MVC.

actionwebservice (0.9.3.3225, 0.9.3)
Web service support for Action Pack.

activerecord (1.13.0.3225, 1.13.0)
Implements the ActiveRecord pattern for ORM.

activesupport (1.2.3.3225, 1.2.3)
Support and utility classes used by the Rails framework.

localization_generator (1.0.8)
[Rails] Localization generator.

rails (0.14.3.3225, 0.14.3)
Web-application framework with template engine, control-flow layer,
and ORM.

rake (0.6.2)
Ruby based make-like utility.

salted_login_generator (1.1.1)
[Rails] Login generator with salted passwords.

sources (0.0.1)
This package provides download sources for remote gem installation

sqlite3-ruby (1.1.0)
SQLite3/Ruby is a module to allow Ruby scripts to interface with a
SQLite3 database.
231:~/CD/railsdemos/globize oracle$

For svn…
I see this:

231:~/CD/railsdemos/globize oracle$
231:~/CD/railsdemos/globize oracle$ which svn
/opt/local/bin/svn
231:~/CD/railsdemos/globize oracle$ svn -v
Subcommand argument required
Type ‘svn help’ for usage.
231:~/CD/railsdemos/globize oracle$ svn help
usage: svn [options] [args]
Subversion command-line client, version 1.2.3.
Type ‘svn help ’ for help on a specific subcommand.

Most subcommands take file and/or directory arguments, recursing
on the directories. If no arguments are supplied to such a
command, it recurses on the current directory (inclusive) by default.

Available subcommands:
add
blame (praise, annotate, ann)
cat
checkout (co)
cleanup
commit (ci)
copy (cp)
delete (del, remove, rm)
diff (di)
export
help (?, h)
import
info
list (ls)
lock
log
merge
mkdir
move (mv, rename, ren)
propdel (pdel, pd)
propedit (pedit, pe)
propget (pget, pg)
proplist (plist, pl)
propset (pset, ps)
resolved
revert
status (stat, st)
switch (sw)
unlock
update (up)

Subversion is a tool for version control.
For additional information, see http://subversion.tigris.org/
231:~/CD/railsdemos/globize oracle$

I checked
http://subversion.tigris.org/

The 1.2.3 I have looks recent.

Anyone have any clues for me?

-Dan

11

IIRC, I you need Ruby 1.8.3 as it updated OpenURI.

Kyle

12

Dan, I’d suggest trying to install any of the other plugins using
“script/plugin install”. I’m pretty sure the problem has to do with your
system and is not specific to the Globalize plugin. I just tried it on
mine, and it worked fine.

The first thing you might want to try is to load it manually with svn,
as Anatol suggested earlier:

cd vendor/plugins
svn co svn://www.diluvia.net/rails-plugins/globalize/trunk globalize

Josh Harvey
http://globalize.diluvia.net/wiki

dan.bikle wrote:

Anyone have any clues for me?

-Dan

13

People,

I fixed the problem.
The method which loads the .yaml files which contain the translations
is located here:
lib/localization.rb

Here is some syntax from the method:

def self.load_localized_strings
# Load language files
Dir[RAILS_ROOT + ‘/lang/*.yaml’].each do |filename|
filename =~ /(([a-z]+_?)+).yaml$/
hash = YAML::load(File.read(filename))
file_charset = hash[‘file_charset’] || ‘ascii’

I made a small change and my problems evaporated:

def self.load_localized_strings
# Load language files
Dir[RAILS_ROOT + ‘/lang/*.yaml’].each do |filename|
filename =~ /(([a-z]+_?)+).yaml$/
hash = YAML::load(File.read(filename))
file_charset = hash[‘file_charset’] || ‘utf-8’

-Dan

14

No,

I think that utf-8 makes a better default than ascii.
If I need to deal with something a bit different like big5 or euc-kr,
then I’d put a file_charset k-v-pair in the .yaml.

-Dan

15

Dan B. wrote:

I made a small change and my problems evaporated:

def self.load_localized_strings
# Load language files
Dir[RAILS_ROOT + ‘/lang/*.yaml’].each do |filename|
filename =~ /(([a-z]+_?)+).yaml$/
hash = YAML::load(File.read(filename))
file_charset = hash[‘file_charset’] || ‘utf-8’

Better solution: insert the following “translation” in the yaml file:

file_charset: utf-8