Does anyone work with iso-8859-1 database?

Arnaud_G · July 12, 2006, 4:45pm

hello,

Our database is in is-8859-1, and I want to update some text fields
without success due to some accentuate characters éà ect …

In my html page (where the charset is iso-8859-19) my textarea

display the accentuate characters well and when the user post the form
… I thought that I just need to save it … without success since
ruby map one byte for one character …
So I thought I just need to set the charset to UTF-8 in my html page
and convert to iso in my controller with the iconv library => no success

I also saw some hacks for UTF-8, but I don’t know how to use it for
iso-8859-1
Peak Obsession

=> how do you do to update iso-8859-1database text field ?

thanks
arnaud

Arnaud_G · July 13, 2006, 12:11am

Arnaud,

are you posting the updates with Ajax? Rails uses UTF-8 for Ajax
requests, so you need to convert it to iso-8859-1. Check out this
article for info on how to do it:

http://dema.ruby.com.br/articles/2005/07/22/playing-nice-with-ajax-and-iso-8859-1

/ CJ

Arnaud_G · July 13, 2006, 10:17am

Hi Carl and thanks for tip
To answer your question I do not use ajax for posting …

Do you mean that ajax use UTF8 for posting and “normal” post not ?
(simple ajax dummie question ;-), how do you do an ajax post ? ;-))

I look at the article you give me it works with iconv by converting the
string form utf8 to iso …
I did the same thing directly on my rails app by converting params hash
… without success …
So the problem is what are the default ruby encoding ? I search and find
that there is no default encoding in ruby each byte is map to the string
object …=> So it is not possible to do it directly in rails due to
ruby (I saw that it will be different in ruby 2.0 …)

=> So no solution with ruby ? do you have any feedback about using jcode
and the $KCODE ?

thanks

arnaud

Carl-Johan K. a écrit :

Arnaud_G · July 13, 2006, 11:45am

Hi,

I am currently working with a ISO-8859-9 database…
I am a new member of the group and did not read the rest of the
discussion…

But, let me share what I did how to work with a non-UTF8 database and
ajax…

First, you have to modify
…ruby\lib\ruby\gems\1.8\gems\activesupport-1.3.1\lib\active_support\json\encoders\core.rb
file …

Find method “define_encoder String do |string|” (in my case line 22)
And put this 2 lines of code at the start of method :

    iconv  = Iconv.new('UTF-8', 'ISO-8859-9')
    string = iconv.iconv(string)

of course, you must add require ‘iconv’ at the begining of core.rb…

Then, modify your controller/action methods accepting requests from ajax
calls to reverse the encoding-change… (this time from UTF8 to
ISO-8859-X)
I have written a method fort this: (I am using ajax_scaffold generator,
but you can change the code according to your needs…)

def iconvize(params)
iconv = Iconv.new(‘ISO-8859-9’, ‘UTF-8’)
for scaffold_column in Put_Your_Model_Name_Here.scaffold_columns
if params[:personel][:"#{scaffold_column.name}"].class == String
params[:personel][:"#{scaffold_column.name}"] =
iconv.iconv(params[:personel][:"#{scaffold_column.name}"])
end
end
end

And, as I said before make a call to iconvize method at the beginning of
your actions called by ajax, like this:

def update
begin
iconvize(params)
@personel = Personel.find(params[:id])
…

And, that’s all…
Hope, this helps…

Arnaud_G · July 14, 2006, 9:59am

Hi Erhan,

I try to use the same “hack” you did, without success since rails
never go throw define_encoder in core.rb file
Since I have to gems install one in /usr/lib and one in /usr/local/lib I
did in both without success …
more funny I write wrong syntax in this function, and there was nothing
… no error, it really seems that rails never use this .

any idea ?

arnaud

Erhan ERÝÞKEN a écrit :

Arnaud_G · July 14, 2006, 10:49am

Hi Arnaud,

First of all, I use rails in Windows environment… (I dont know whether
it makes a difference…)

What I suggest you, remove all the “hack” lines from core.rb, and just
make an ajax call from your app… Then, look at your “development.log”
file…

You must see the error trace at the very end of the log file…

This is the way I found that rails makes a call to core.rb…

To help you, here I copy my development.log trace when an ajax call is
made without the “hack”…

You can see the “malformed UTF-8 char…” error, and the call to
“unpack” in core.rb file in the trace.

When I do the “hack”, all these errors went away, it renders the .rjs
files correctly and updates the iso-8859-9 database with correct Turkish
chars.

Also, after you did what I said to you above (remove hack, make an ajax
call, look at log file), you can send me your error trace in
development.log file, and we can look for what’s happening in your case
together…

Here is my error trace :

ActionView::TemplateError (malformed UTF-8 character) on line #6 of
app/views/personel/edit.rjs:
3:
4: if @successful
5: page.hide element_row_id(@view_options)
6: page.insert_html :bottom, scaffold_tbody_id(@options), :partial =>
‘new_edit’
7: page << “new TableRow.MoveAfter(’#{element_row_id(@view_options)}’,
‘#{element_row_id(@options)}’);”
8: page.show element_row_id(@options)
9: page << “setupAllTabs()”
D:/EErisken/Downloads/ruby/InstantRails/ruby/lib/ruby/gems/1.8/gems/activesupport-1.3.1/lib/active_support/json/encoders/core.rb:36:in
`unpack' D:/EErisken/Downloads/ruby/InstantRails/ruby/lib/ruby/gems/1.8/gems/activesupport-1.3.1/lib/active_support/json/encoders/core.rb:36 D:/EErisken/Downloads/ruby/InstantRails/ruby/lib/ruby/1.8/jcode.rb:212:in`each_char’
D:/EErisken/Downloads/ruby/InstantRails/ruby/lib/ruby/1.8/jcode.rb:211:in
`each_char' D:/EErisken/Downloads/ruby/InstantRails/ruby/lib/ruby/gems/1.8/gems/activesupport-1.3.1/lib/active_support/json/encoders/core.rb:27 D:/EErisken/Downloads/ruby/InstantRails/ruby/lib/ruby/gems/1.8/gems/activesupport-1.3.1/lib/active_support/core_ext/object/misc.rb:23:in`returning’
D:/EErisken/Downloads/ruby/InstantRails/ruby/lib/ruby/gems/1.8/gems/activesupport-1.3.1/lib/active_support/json/encoders/core.rb:26

Bu e-posta mesaji, mesajin alici kisminda belirtilmis olan kullanici
icindir. Mesajin alicisi siz degilseniz dogrudan veya dolayli olarak
mesaji kullanmayiniz, acmayiniz, dagitmayiniz, yazicidan dokumunu
almayiniz veya herhangi bir kismini kopyalamayiniz. Yanlislikla bu mesaj
size ulasmissa lutfen, siliniz ve tum kopyalarini yok ederek mesaji
gonderene acilen haber veriniz. Bu mesaj icerisinde belirtilenler sadece
gondericinin kisisel gorusleridir. Bu gorusler Sermaye Piyasasi Kurulu’
nun (SPK) goruslerini yansitmadigi gibi, SPK’ yi baglayici da degildir.
Bu mesajin icerisinde ya da eklerinde yer alan bilgilerin dogrulugu,
butunlugu ve guncelligi SPK tarafindan garanti edilmemektedir ve bilinen
viruslere karsi kontrolleri yapilmis olarak yollanan mesajin sitenizde
yaratabilecegi zararlardan SPK sorumlu tutulamaz.

This email is intended solely for the use of the individual or entity to
whom it is adressed. If you are not the intended addressee of this
message, you should not use, open, disseminate, distrubute, print or
copy this e-mail. If you have received this email in error, please
delete it from your system and notify the sender immediately. The
Capital Markets Board of Turkey (CMB) does not accept any legal
responsibility whatsoever for the contents of this message. Any opinions
contained in this message are those of the author and are not given or
endorsed by the CMB. The CMB does not warrant the accuracy, integrity
and currency of the information transmitted with this message. This
message has been detected for all known computer viruses thence CMB is
not liable for the occurrence of any system corruption caused by this
message.

Arnaud_G · July 14, 2006, 10:03pm

Hi Arnaud,

by the way, a question came to my mind, have you restart the web-server
after you made the hack?

Kimden: Arnaud G. [mailto:[email protected]]
GÃ¶nderilmiÅ?: Cum 14.07.2006 10:58
Kime: [email protected]
Bilgi: Erhan ERÄ°Å?KEN
Konu: Re: [Rails] Does anyone work with iso-8859-1 database ?

Hi Erhan,

I try to use the same “hack” you did, without success since rails
never go throw define_encoder in core.rb file
Since I have to gems install one in /usr/lib and one in /usr/local/lib I
did in both without success …
more funny I write wrong syntax in this function, and there was nothing
… no error, it really seems that rails never use this .

any idea ?

arnaud

And put this 2 lines of code at the start of method :
iconv = Iconv.new(‘ISO-8859-9’, ‘UTF-8’)
begin
Sent: Thursday, July 13, 2006 11:16 AM
string form utf8 to iso …

Arnaud,
On 7/12/06, Arnaud G. [email protected] wrote:

display the accentuate characters well and when the user post the form

[email protected]

Rails mailing list
[email protected]
http://lists.rubyonrails.org/mailman/listinfo/rails

Bu e-posta mesaji, mesajin alici kisminda belirtilmis olan kullanici
icindir. Mesajin alicisi siz degilseniz dogrudan veya dolayli olarak
mesaji kullanmayiniz, acmayiniz, dagitmayiniz, yazicidan dokumunu
almayiniz veya herhangi bir kismini kopyalamayiniz. Yanlislikla bu mesaj
size ulasmissa lutfen, siliniz ve tum kopyalarini yok ederek mesaji
gonderene acilen haber veriniz. Bu mesaj icerisinde belirtilenler sadece
gondericinin kisisel gorusleridir. Bu gorusler Sermaye Piyasasi Kurulu’
nun (SPK) goruslerini yansitmadigi gibi, SPK’ yi baglayici da degildir.
Bu mesajin icerisinde ya da eklerinde yer alan bilgilerin dogrulugu,
butunlugu ve guncelligi SPK tarafindan garanti edilmemektedir ve bilinen
viruslere karsi kontrolleri yapilmis olarak yollanan mesajin sitenizde
yaratabilecegi zararlardan SPK sorumlu tutulamaz.

This email is intended solely for the use of the individual or entity to
whom it is adressed. If you are not the intended addressee of this
message, you should not use, open, disseminate, distrubute, print or
copy this e-mail. If you have received this email in error, please
delete it from your system and notify the sender immediately. The
Capital Markets Board of Turkey (CMB) does not accept any legal
responsibility whatsoever for the contents of this message. Any opinions
contained in this message are those of the author and are not given or
endorsed by the CMB. The CMB does not warrant the accuracy, integrity
and currency of the information transmitted with this message. This
message has been detected for all known computer viruses thence CMB is
not liable for the occurrence of any system corruption caused by this
message.

Arnaud_G · July 13, 2006, 2:57pm

Many thanks Erhan for your help, it will work like this …
So the global idea is always to use ajax(which work with utf-8) before
converting with iconv in iso-8859-1.

arnaud

Erhan ERÝÞKEN a écrit :