[Feature:1.9] transcode for UTF8-MAC


#1

e$B$3$s$P$s$Oe(B sheepman e$B$G$9!#e(B

1.9 e$B$Ne(B TODO
e$B%j%9%H$,$I$N$h$&$K4IM}$5$l$F$$$k$N$+J,$+$i$J$$$N$Ge(B
e$B3NG’$G$9!#e(B

UTF8-MACe$B$Ne(Btranscodee$B$,<BAu$5$l$kM=Dj$H$$$&$N$O$"$k$N$G$7$g$&$+!#e(B
e$B$D$^$je(B

“e$B%F%9%He(B”.encode(“UTF8-MAC”)

e$B$,F0:n$9$k$h$&$K$J$kM=Dj$O$"$k$N$G$7$g$&$+!#e(B

(1)e$BM=Dj$"$je(B
(2)e$B$d$j$?$$$1$I?M<j$H;~4V$,$J$$$N$Ge(B1.9.2e$B$G$OL5M}$=$&!#e(B
(3)e$B$d$i$J$$e(B

e$B$N$I$l$G$7$g$&$+!#e(B

e$B$^$?!Xe(BRuby M17N e$B$N@_7W$H<BAu!Y$Ne(B
http://jp.rubyist.net/magazine/?0025-Ruby19_m17n#l70
Mac OS Xe$B$K4X$9$k0J2<$NItJ,$Oe(B

e$B0lJ}!"%U%!%$%kL>$rEO$9:]$O!“EO$5$l$?J8;zNs$N%(%s%3!<%G%#%s%0$,e(B ASCII-8BIT
e$B$J$i$P%P%$%HNs$H$_$J$7$F$=$N$^$^%7%9%F%$K!"$=$l0J30$G$Oe(B UTF8-MAC e$B$X$HJQ49e(B e$B$7$?>e$G%7%9%F%$KEO$7$^$9!#$J$*!”$3$NF0:n$O8!F$$,IT==J,$J$?$a!">-MhJQ99$,e(B
e$B9T$o$l$k2DG=@-$,$"$j$^$9e(B (Unix e$B$KF0:n$O9g$o$;$k$J$Ie(B)e$B!#e(B

e$B>-Mh$N$"$k$Y$-;EMM$r@bL@$7$F$$$k$H$$$&2r<a$GNI$$$G$7$g$&$+!#e(B
e$B8=:_$N$H$3$me(B UTF8-MAC e$B$X$OJQ49$5$l$:Nc30$,H/@8$9$k$h$&$G$9!#e(B

e$B;29Me(B)
http://www.kumaryu.net/cgi-bin/diary/?date=20090222#p02


#2

e$B$J$+$@$G$9!#e(B

At Thu, 16 Apr 2009 22:21:24 +0900,
NARUSE, Yui wrote in [ruby-dev:38310]:

e$B$o$?$7$Oe(BMac OS Xe$B$G@83h$7$F$$$k$o$1$G$O$J$$$N$G!“e(B(e$B4D6-$O;}$C$F$k$s$G$9$,e(B)
e$B$I$&$9$k$N$,e(BMac OS Xe$B$GJk$i$7$F$$$k?M$K$H$C$F9,$;$J$N$+$o$+$i$J$$$N$G$9!#e(B
e$B$J$N$G!”!Ve(BUnixe$B$KF0:n$r9g$o$;$?J}$,9,$;$J$N$@$m$&$+!"$=$l$H$b:#$N$[$&$,!&!&!&!We(B
e$B$HLB$C$F$$$k$H$$$&$N$,@53N$J$H$3$m$G$9!#e(B

Mac OS
Xe$B$N%7%9%F%`%3!<%k$H$7$F$O!"e(BNFCe$B$@$m$&$He(BNFDe$B$@$m$&$H6hJL$;$:e(B
e$B$KDL$9$h$&$G$9!#e(B

e$B8=>u$G$O!"%7%’%k$Ne(Bglobe$B$G$Oe(BNFDe$B$,$=$N$^$^EO$5$l$?$j!"$=$l0J30$NIte(B
e$BJ,$G$Oe(BNFCe$B$K$J$C$F$$$?$j!“e(BARGVe$B$de(BENVe$B$Oe(BUTF-8e$B$N$^$^$G$”$C$?$j!“e(B
UTF8-MACe$B$rJ,$1$F$$$k$3$H$K$h$k%a%j%C%H$O$”$j$^$;$s!#e(BUTF-8e$B$KE}0le(B
e$B$7$F!"e(Bnormalizee$B$9$k%a%=%C%I$rDI2C$9$k$[$&$,$h$$$N$G$O$J$$$G$7$ge(B
e$B$&$+!#e(B


#3

e$B@.@%$G$9!#e(B

Nobuyoshi N. wrote:

Mac OS Xe$B$N%7%9%F%`%3!<%k$H$7$F$O!"e(BNFCe$B$@$m$&$He(BNFDe$B$@$m$&$H6hJL$;$:e(B
e$B$KDL$9$h$&$G$9!#e(B

e$B@53N$K$Oe(BVFSe$B%I%i%$%P$N%l%$%d!<$G$9$M!#e(B
e$B$=$3$G!">e0L%l%$%d!<$+$i$N%Q%9$O%U%!%$%k%7%9%F%$KE,$7$?7A$K!"e(B e$B%U%!%$%k%7%9%F%$+$i$N%Q%9$Oe(Bdecomposed
UTF-8e$B$K!"4pK\E*$K$OJQ49$7$^$9!#e(B

e$B!V%U%!%$%k%7%9%F%`$KE,$7$?7A!W$H$$$&$N$O!"I,$:$7$be(Bdecomposed
Unicodee$B$K$Oe(B
e$B$J$i$J$$$H$$$&0UL#$G$9!#e(B
e$BNc$($P!"e(BHFS+e$B$G$Oe(Bdecomposed UTF-16e$B$K$J$j$^$9!#e(B
e$B$7$+$7!“e(Bsmbe$B7PM3$Ge(BWindowse$B$N%U%!%$%k%5!<%P!<$K%”%/%;%9$9$k>l9g!“e(B
Mac OS Xe$B$O%U%!%$%kL>$r9g@.:Q$7A<0$K$7$F%"%/%;%9$r9T$$$^$9!#e(B
e$B$=$7$F!"J,2r:Q$
$N%U%!%$%kL>$r;}$D%U%!%$%k$K$O%”%/%;%9$9$k$3$H$,$G$-$^$;$s!#e(B

e$BNc30$,e(BUFSe$B$He(BNFSe$B$G!"$3$Ne(B2e$B$D$G$O>e5-$N=hM}$,9T$o$l$^$;$s!#e(B

e$B8=>u$G$O!"%7%’%k$Ne(Bglobe$B$G$Oe(BNFDe$B$,$=$N$^$^EO$5$l$?$j!"$=$l0J30$NIte(B
e$BJ,$G$Oe(BNFCe$B$K$J$C$F$$$?$j!“e(BARGVe$B$de(BENVe$B$Oe(BUTF-8e$B$N$^$^$G$”$C$?$j!“e(B
UTF8-MACe$B$rJ,$1$F$$$k$3$H$K$h$k%a%j%C%H$O$”$j$^$;$s!#e(BUTF-8e$B$KE}0le(B
e$B$7$F!"e(Bnormalizee$B$9$k%a%=%C%I$rDI2C$9$k$[$&$,$h$$$N$G$O$J$$$G$7$ge(B
e$B$&$+!#e(B

e$B!Ve(Bnormalizee$B$9$k%a%=%C%I!W$Oe(Br23296e$B$K$F!"e(Btranscodee$B$Ke(BUTF8-MACe$B$rDI2C$H$$$&7A$Ge(B
e$BBP=h$r9T$$$^$7$?!#e(B

e$B!Ve(BUTF-8e$B$KE}0l!W$H$$$&$N$K$O@_Dj$9$k%(%s%3!<%G%#%s%0$r!Ve(BUTF-8e$B!W$K$9$ke(B
e$B$H$$$&0UL#$@$1$G$9$+!)e(B
e$B%U%!%$%kL>$N%j%9%H$r<hF@$9$k:]$KJQ49$H$$$&OC$b$J$5$C$F$$$?$h$&$K;W$$$^$9$,!#e(B

e$B$G!"$=$N>l9ge(BARGVe$B$de(BENVe$B$O$I$&$9$k$N$+$J$!$H$$$&5?Ld$,$"$ke(B


#4

e$B@.@%$G$9!#e(B

sheepman wrote:

UTF8-MACe$B$Ne(Btranscodee$B$,<BAu$5$l$kM=Dj$H$$$&$N$O$"$k$N$G$7$g$&$+!#e(B
(1)e$BM=Dj$"$je(B
(2)e$B$d$j$?$$$1$I?M<j$H;~4V$,$J$$$N$Ge(B1.9.2e$B$G$OL5M}$=$&!#e(B
(3)e$B$d$i$J$$e(B

e$B$N$I$l$G$7$g$&$+!#e(B

(1)e$B$He(B(2)e$B$N4V$G!"$d$j$?$$$1$l$Ie(B1.9.2e$B$KF~$k$+$O$o$+$i$J$$!"$G$9!#e(B
e$B$I$&$7$^$7$g$&$+$M!)!de(BMartine$B$5$se(B

Unicodee$B@55,2=$NOHAH$_$r<BAu$7$?$&$($G!"$3$NJQ49$r$$$l$k$+!“e(B
e$B$3$3$@$1@h9T$7$F<BAu$7$F$7$^$&$+$H$+$NH=CG$b$”$j$D$D!#e(B
e$B7k6I$3$NJU$O$I$NDxEY$N%$%s%Q%/%H$NLdBj$G$"$j!"e(B
e$B$I$N$h$&$J$b$N$,K>$^$l$F$$$k$+$,$o$+$i$J$$$HF0$-$E$i$$$N$G!“e(B
e$B$G$-$l$P8+$($kHO0O$G@<$r>e$2$F$$$?$@$-$?$$$H$3$m$G$”$j$^$9!#e(B

e$B8=:_$N$H$3$me(B UTF8-MAC e$B$X$OJQ49$5$l$:Nc30$,H/@8$9$k$h$&$G$9!#e(B
e$B$o$?$7$Oe(BMac OS
Xe$B$G@83h$7$F$$$k$o$1$G$O$J$$$N$G!“e(B(e$B4D6-$O;}$C$F$k$s$G$9$,e(B)
e$B$I$&$9$k$N$,e(BMac OS
Xe$B$GJk$i$7$F$$$k?M$K$H$C$F9,$;$J$N$+$o$+$i$J$$$N$G$9!#e(B
e$B$J$N$G!”!Ve(BUnixe$B$KF0:n$r9g$o$;$?J}$,9,$;$J$N$@$m$&$+!"$=$l$H$b:#$N$[$&$,!&!&!&!We(B
e$B$HLB$C$F$$$k$H$$$&$N$,@53N$J$H$3$m$G$9!#e(B

e$B>e5-$NItJ,$O!";EMM$,L$3NDj$G$"$k;]$NCm0U=q$-$G$"$k$H2r<a$7$F$/$@$5$$!#e(B

e$B<c430c$&OC$G$O$"$j$^$9$,!"$?$H$($PF|K\8le(BWindowse$B$N>l9g$O!“e(B
Shift_JISe$B0!<o$G$”$ke(BCP932e$B$r40A4$KJLJ*07$$$K$7$F$$$^$9$,!"e(B
e$B$3$3$K$$$?$k$^$G$O0$I!6+4-$NCf$+$iCN8+$r=&$$>e$2$F7hDj$7$F$$$k$o$1$G$9!#e(B
Mac OS Xe$B$K$D$$$F$O$^$@$^$@HaLD$,B-$j$^$;$s!#e(B

e$B;29Me(B)
http://www.kumaryu.net/cgi-bin/diary/?date=20090222#p02

e$B>R2p$"$j$,$H$&$4$6$$$^$9!#e(B
e$B8+$F$_$^$9!#e(B


#5

e$B@.@%$G$9!#e(B

Nobuyoshi N. wrote:

e$BNc30$,e(BUFSe$B$He(BNFSe$B$G!"$3$Ne(B2e$B$D$G$O>e5-$N=hM}$,9T$o$l$^$;$s!#e(B

e$B%U%!%$%k%7%9%F%`$r0U<1$7$?%3!<%I$r=q$+$J$1$l$P$J$i$J$$$H$$$&$N$Oe(B
e$B$+$J$j%$%d$G$9$M$’!#e(B

e$B?7$7$$e(BFSe$B$,=P$F$-$?;~$K$=$3$G$O$I$&$$$&07$$$r$9$k$N$+!"e(B
e$B$H$+$b9M$($J$$$H$$$1$^$;$s$+$i$M$’!#e(B

e$BA0<T$O$^$!$H$b$+$/!"8e<T$OE~Dl;H$$$d$9$$$H$O;W$($^$;$s!#$;$a$Fe(B
encode(“UTF-8”, normalize: :decompose) e$B$H$+!#e(B

e$B8e<T$KBP$9$kLdBj0U<1$OM}2r$7$^$9$,!";XDj$N:]$K$O9T$&JQ49$,!“e(B
e$B!Ve(BApplee$B$K$h$C$F=$@5$5$l$?e(BNFDe$B$+$iLa$9$?$a$NJQ49!W$@$HL@<($9$kI,MW$,$”$j$^$9!#e(B
e$B$N$G!"e(Bencode(“UTF-8”, normalize:
:decompose_mac)e$B$H$+$K$J$j$^$9$+!#e(B

e$B$7$+$7!"$$$C$=$N$3$He(BUTF8-MACe$B$re(BUTF-8e$B$Ne(Baliase$B$K$7$A$c$C$F$b$$$$$+$J$H$b;W$C$F$$$^$9!#e(B

e$B!Ve(BUTF-8e$B$KE}0l!W$H$$$&$N$K$O@_Dj$9$k%(%s%3!<%G%#%s%0$r!Ve(BUTF-8e$B!W$K$9$ke(B
e$B$H$$$&0UL#$@$1$G$9$+!)e(B
e$B%U%!%$%kL>$N%j%9%H$r<hF@$9$k:]$KJQ49$H$$$&OC$b$J$5$C$F$$$?$h$&$K;W$$$^$9$,!#e(B

e$B$G!"$=$N>l9ge(BARGVe$B$de(BENVe$B$O$I$&$9$k$N$+$J$!$H$$$&5?Ld$,$"$ke(B

e$B<+F0E*$JJQ49$O0l@Z$J$7!"$H$$$&J}8~$K79$$$F$$$^$9!#e(B

e$B<+F0JQ49$rF~$l$k$H>e=R$N%U%!%$%k%7%9%F%`$4$H$KJ,4t$9$k$+H]$+$H$+$b$"$k$s$G$9$h$M!#e(B
e$B0lIt$r<h$j=P$;$P$"$C$?J}$,NI$5$=$&$@$,!“e(B
e$BA4BN$H$7$FL7=b$N$J$$=hM}$,$-$o$a$F:$Fq$J$N$GJQ49$J$7!”$K$J$k$+$J$!$H!#e(B


#6

e$B@.@%$G$9!#e(B

NARUSE, Yui wrote:

e$B?7$7$$e(BFSe$B$,=P$F$-$?;~$K$=$3$G$O$I$&$$$&07$$$r$9$k$N$+!"e(B
e$B$H$+$b9M$($J$$$H$$$1$^$;$s$+$i$M$’!#e(B

e$B$=$&$$$($P!“e(BSnowLeoparde$B$Ge(BZFSe$B$NFI$_=q$-$,%5%]!<%H$5$l$ke(B
(Leoparde$B$G$b%5!<%P!<HG$OFI$9~$%5%]!<%H$,$”$ke(B)e$B$HJ9$$$?$N$G$9$,!"e(B
e$B<B:]$K$3$NJU$,$I$&$$$&F0$-$K$J$k$N$+$4B8$8$JJ}$O65$($F$/$@$5$$!#e(B

encode(“UTF-8”, “UTF8-MAC”) e$B$He(B
encode(“UTF8-MAC”).force_encoding(“UTF-8”) e$B$G$9$+e(B?

e$BA0<T$O$^$!$H$b$+$/!"8e<T$OE~Dl;H$$$d$9$$$H$O;W$($^$;$s!#$;$a$Fe(B
encode(“UTF-8”, normalize: :decompose) e$B$H$+!#e(B

e$B8e<T$KBP$9$kLdBj0U<1$OM}2r$7$^$9$,!";XDj$N:]$K$O9T$&JQ49$,!“e(B
e$B!Ve(BApplee$B$K$h$C$F=$@5$5$l$?e(BNFDe$B$+$iLa$9$?$a$NJQ49!W$@$HL@<($9$kI,MW$,$”$j$^$9!#e(B
e$B$N$G!"e(Bencode(“UTF-8”, normalize: :decompose_mac)e$B$H$+$K$J$j$^$9$+!#e(B

Normalizee$B$Ne(BAPIe$B$r$I$&$9$k$N$+$O0JA0$b$b$a$?5$$,$9$k$N$G!"$H$j$"$($:@hAw$j$G!#e(B

e$B$H$j$"$($:!“e(Br23308e$B$+$ie(BMac OS Xe$B$K$$$$F!"e(B
filesystem_encodinge$B$re(BUTF-8e$B$X$HJQ99$7$F$$$^$9!#e(B
e$B$J$
!”<+F0JQ49$K$D$$$F$O:#$^$GDL$j9T$C$F$$$^$9!#e(B


#7

e$B$J$+$@$G$9!#e(B

At Wed, 29 Apr 2009 18:43:26 +0900,
NARUSE, Yui wrote in [ruby-dev:38356]:

e$B!V%U%!%$%k%7%9%F%`$KE,$7$?7A!W$H$$$&$N$O!"I,$:$7$be(Bdecomposed Unicodee$B$K$Oe(B
e$B$J$i$J$$$H$$$&0UL#$G$9!#e(B
e$BNc$($P!"e(BHFS+e$B$G$Oe(Bdecomposed UTF-16e$B$K$J$j$^$9!#e(B
e$B$7$+$7!“e(Bsmbe$B7PM3$Ge(BWindowse$B$N%U%!%$%k%5!<%P!<$K%”%/%;%9$9$k>l9g!“e(B
Mac OS Xe$B$O%U%!%$%kL>$r9g@.:Q$7A<0$K$7$F%"%/%;%9$r9T$$$^$9!#e(B
e$B$=$7$F!"J,2r:Q$
$N%U%!%$%kL>$r;}$D%U%!%$%k$K$O%”%/%;%9$9$k$3$H$,$G$-$^$;$s!#e(B

e$BNc30$,e(BUFSe$B$He(BNFSe$B$G!"$3$Ne(B2e$B$D$G$O>e5-$N=hM}$,9T$o$l$^$;$s!#e(B

e$B%U%!%$%k%7%9%F%`$r0U<1$7$?%3!<%I$r=q$+$J$1$l$P$J$i$J$$$H$$$&$N$Oe(B
e$B$+$J$j%$%d$G$9$M$’!#e(B

e$B8=>u$G$O!"%7%’%k$Ne(Bglobe$B$G$Oe(BNFDe$B$,$=$N$^$^EO$5$l$?$j!"$=$l0J30$NIte(B
e$BJ,$G$Oe(BNFCe$B$K$J$C$F$$$?$j!“e(BARGVe$B$de(BENVe$B$Oe(BUTF-8e$B$N$^$^$G$”$C$?$j!“e(B
UTF8-MACe$B$rJ,$1$F$$$k$3$H$K$h$k%a%j%C%H$O$”$j$^$;$s!#e(BUTF-8e$B$KE}0le(B
e$B$7$F!"e(Bnormalizee$B$9$k%a%=%C%I$rDI2C$9$k$[$&$,$h$$$N$G$O$J$$$G$7$ge(B
e$B$&$+!#e(B

e$B!Ve(Bnormalizee$B$9$k%a%=%C%I!W$Oe(Br23296e$B$K$F!"e(Btranscodee$B$Ke(BUTF8-MACe$B$rDI2C$H$$$&7A$Ge(B
e$BBP=h$r9T$$$^$7$?!#e(B

encode(“UTF-8”, “UTF8-MAC”) e$B$He(B
encode(“UTF8-MAC”).force_encoding(“UTF-8”) e$B$G$9$+e(B?

e$BA0<T$O$^$!$H$b$+$/!"8e<T$OE~Dl;H$$$d$9$$$H$O;W$($^$;$s!#$;$a$Fe(B
encode(“UTF-8”, normalize: :decompose) e$B$H$+!#e(B

e$B!Ve(BUTF-8e$B$KE}0l!W$H$$$&$N$K$O@_Dj$9$k%(%s%3!<%G%#%s%0$r!Ve(BUTF-8e$B!W$K$9$ke(B
e$B$H$$$&0UL#$@$1$G$9$+!)e(B
e$B%U%!%$%kL>$N%j%9%H$r<hF@$9$k:]$KJQ49$H$$$&OC$b$J$5$C$F$$$?$h$&$K;W$$$^$9$,!#e(B

e$B$G!"$=$N>l9ge(BARGVe$B$de(BENVe$B$O$I$&$9$k$N$+$J$!$H$$$&5?Ld$,$"$ke(B

e$B<+F0E*$JJQ49$O0l@Z$J$7!"$H$$$&J}8~$K79$$$F$$$^$9!#e(B


#8

e$B%A%1%C%He(B #1411 e$B$,99?7$5$l$^$7$?!#e(B (by Yui NARUSE)

e$B%9%F!<%?%9e(B Opene$B$+$ie(BClosede$B$KJQ99e(B

e$B$3$l$C$Fe(Bclosee$B$G$$$$$G$9$h$Me(B

http://redmine.ruby-lang.org/issues/show/1411