Replacing diacritics by simple character

Une_BSSSSvS_ue · September 25, 2007, 6:31pm

do u know of a way to replace diacritics by simple character (ie. : é
-o-> e)

the same with ligatures (ie. : Æ -o-> AE )

using tables ?

Une_BSSSSvS_ue · September 25, 2007, 6:57pm

Le 25 septembre à 18:25, Une Bévue a écrit :

(Hello again… )

do u know of a way to replace diacritics by simple character (ie. : é
-o-> e)

the same with ligatures (ie. : Æ -o-> AE )

using tables ?

IConv can do that for you :

require “iconv”
=> true

i = Iconv.new(“ASCII//TRANSLIT”, “ISO-8859-15”)
=> #Iconv:0x84d4448

i.iconv(“aéouï Æ”)
=> “a’eou"i AE”

i.iconv(“aéouï Æ”).gsub(/[^a-zA-Z0-9 ]/, ‘’)
=> “aeoui AE”

Fred

Une_BSSSSvS_ue · September 25, 2007, 8:13pm

–
I’ve found an axe can do a lot for a paper-mangling printer. Especially
if you shout for one at the top of your voice, and then a cow orker
--------------------------------------------------------------------------------------^
???
brings you said instrument. Suddenly, no more paper jams.
(Kai Henningsen in the SDM)

Une_BSSSSvS_ue · September 25, 2007, 9:01pm

F. Senault [email protected] wrote:

IConv can do that for you :

require “iconv”
=> true
i = Iconv.new(“ASCII//TRANSLIT”, “ISO-8859-15”)
=> #Iconv:0x84d4448
i.iconv(“aéouï Æ”)
=> “a’eou"i AE”
i.iconv(“aéouï Æ”).gsub(/[^a-zA-Z0-9 ]/, ‘’)
=> “aeoui AE”

Fine thanks a lot Fred à c’t’heure

Have a good wine celler

ça marche même avec de l’UTF-8

works also with UTF-8

Une_BSSSSvS_ue · September 25, 2007, 9:43pm

Le 25 septembre à 20:12, Michal S. a écrit :

–
I’ve found an axe can do a lot for a paper-mangling printer. Especially
if you shout for one at the top of your voice, and then a cow orker
--------------------------------------------------------------------------------------^
???

It’s intentional. Cow orker was probably a typo in the olden times, but
has entered the mainstream since then. Just ask google : “Results 1 -
10 of about 37,200 for “cow orker”. (0.19 seconds)”

Fred

Une_BSSSSvS_ue · September 25, 2007, 9:26pm

On Sep 25, 2007, at 18:55, F. Senault wrote:

do u know of a way to replace diacritics by simple character (ie. : Ã©
-o-> e)

the same with ligatures (ie. : Ã† -o-> AE )

using tables ?

IConv can do that for you :

An alternative approach is something like Sean M. Burke’s
Text::Unidecode:

http://interglacial.com/~sburke/tpj/as_html/tpj22.html

Here is an example of an implementation of Unidecode in Lua [1]:

local Unidecode = require( ‘Unidecode’ )

print( Unidecode( ‘ÐœÐ¾ÑÐºÐ²Ð°Ì’ ) )
print( Unidecode( ‘åŒ—äº¬’ ) )
print( Unidecode( ‘á¼ˆÎ¸Î·Î½á¾¶’ ) )
print( Unidecode( ‘ì„œìš¸’ ) )
print( Unidecode( ‘æ±äº¬’ ) )
print( Unidecode( ‘äº¬éƒ½å¸‚’ ) )
print( Unidecode( ‘à¤¨à¥‡à¤ªà¤¾à¤²’ ) )
print( Unidecode( ‘×ªÖµÖ¼×œÖ¾×Ö¸×‘Ö´×™×‘-×™Ö¸×¤×•Ö¹’ ) )
print( Unidecode( ‘ØªÙŽÙ„Ù’ Ø£ÙŽØ¨ÙÙŠØ¨Ù’ ÙŠÙŽØ§ÙÙŽØ§’ ) )
print( Unidecode( ‘ØªÙ‡Ø±Ø§Ù†’ ) )
print( Unidecode( ‘GÃ©ometrie DiffÃ©rentielle’ ) )

Moskva
beijing
Athena
seoul
dongjing
jingdushi
nepaal
te’labiyb-yapvo
tal 'abiyb yaafaa
thran
Geometrie Differentielle

Cheers,

PA.

[1] http://dev.alt.textdrive.com/browser/HTTP/Unidecode.lua

Une_BSSSSvS_ue · September 26, 2007, 12:01am

F. Senault wrote:

IConv can do that for you :

require “iconv”
=> true

i = Iconv.new(“ASCII//TRANSLIT”, “ISO-8859-15”)
=> #Iconv:0x84d4448

i.iconv(“aéouï Æ”)
=> “a’eou"i AE”

i.iconv(“aéouï Æ”).gsub(/[^a-zA-Z0-9 ]/, ‘’)
=> “aeoui AE”

That doesn’t work on all platforms. For me:

require “iconv”
=> true

i = Iconv.new(“ASCII//TRANSLIT”, “UTF-8”)
=> #Iconv:0xb7cf28e0

i.iconv(“aéouï Æ”)
=> “a?ou? AE”

Une_BSSSSvS_ue · September 25, 2007, 9:49pm

How do i get off this mailing list ? THANKS!!!

Une_BSSSSvS_ue · September 26, 2007, 12:25am

Daniel DeLorme [email protected] wrote:

Are u sure about the encoding of “aéouï Æ” ?

because i did it with UTF-8, it works :

– the script ----------------------------------------------------------
#! /usr/bin/env ruby

require “iconv”

i = Iconv.new(“ASCII//TRANSLIT”, “UTF-8”)

p i.iconv(“aéouï Æ”)

=> “a’eou"i AE”

p i.iconv(“aéouï Æ”).gsub(/[^a-zA-Z0-9 ]/, ‘’)

=> “aeoui AE”

p i.iconv(“Être ou ne pas être, c’est la question. aéouï Æ, wie heiß du
?”).gsub(/[^a-zA-Z0-9’ ]/, ‘’).gsub(/[’ ]/, '').gsub(/(.*)$/, ‘\1’)

=> “Etre_ou_ne_pas_etre_c_est_la_question_a_eoui_AE_wie_heiss_du”

p i.iconv(“Être ou ne pas être, c’est la question. aéouï Æ, wie heiß
du?”).gsub(/[^a-zA-Z0-9’ ]/, ‘’).gsub(/[’ ]/, '').gsub(/(.*)$/, ‘\1’)

=> “Etre_ou_ne_pas_etre_c_est_la_question_a_eoui_AE_wie_heiss_du”

Une_BSSSSvS_ue · September 26, 2007, 12:59am

Une Bévue wrote:

Are u sure about the encoding of “aéouï Æ” ?

yep.

str = “aéouï Æ”
=> “a\303\251ou\303\257 \303\206” #(that’s utf8 allright)

i.iconv(str)
=> “a?ou? AE”

but like I said, translit doesn’t work the same on all platforms (I’m on
ubuntu btw)

Daniel

Une_BSSSSvS_ue · September 26, 2007, 2:58am

Daniel DeLorme [email protected] wrote:

but like I said, translit doesn’t work the same on all platforms (I’m on
ubuntu btw)

i’m running Mac OS X 10.4.10…