String#valid_encoding? shoud be strict?

e$B9b66@,5A$G$9!#e(B1.9e$B$N%(%s%3!<%G%#%s%0$He(BString#valid_encoding?e$B$K$D$$$F!#e(B

e$B!ZGX7J![e(B
e$BF~NO%A%’%C%/Ey$G!"$"$kJ8;zNs$,$"$k%(%s%3!<%G%#%s%0$NHO0OFb$Ke(B
e$B$$5$^$C$F$$$k$+!“873J$K%A%’%C%/$7$?$$>l9g$,$”$j$^$9!#e(B
e$B!J$
$5$^$C$F$$$J$$>l9g$O:FF~NO$r$&$J$,$7$?$j!"!Ve(B?e$B!W$K6/@)E*$Ke(B
e$BCV49$7$?$j$7$^$9!K!#e(B

e$B!ZDs0F![e(B
e$B$H$$$&$o$1$G!"Nc$($Pe(B String#valid_encoding?
e$B$G873J$J%A%’%C%/$,e(B
e$B$G$-$k$H$&$l$7$$$G$9!#6qBNE*$K$O!"e(BShift_JISe$B$G!"e(BJISe$B$K$J$$J8;ze(B
e$B!J$$$o$f$k5!<o0MB8J8;z!K$,4^$^$l$F$$$k>l9g!"e(Bfalsee$B$K$J$k$He(B
e$B$$$C$?6q9g$G$9!#e(B

e$B!&%5%s%W%k%3!<%Ie(B

encoding:shift_jis

str1=“e$B$”$$$&$($*e(B"
p str1.encoding #=> Encoding:Shift_JIS
p str1.valid_encoding? #=> true

str2=“e$B$”$$$&$($*-!-"-#e(B"
p str2.encoding #=> Encoding:Shift_JIS
p str2.valid_encoding? #=>
e$B8=>ue(Btruee$B$@$1$I$3$3$re(Bfalsee$B$K$7$?$$e(B

str3=“e$B$”$$$&$($*-!-"-#e(B"
str3.force_encoding(“Windows-31J”)
p str3.encoding #=> Encoding:Windows-31J
p str3.valid_encoding? #=> true e$B$N$^$^$Ge(BOK

e$B!ZHw9M![e(B
e$B$3$l$C$F!“e(BWindows-31Je$B$,e(BShift_JISe$B$Ne(Breplicae$B$=$N$^$^$G$$$$$N$+e(B
e$B$I$&$+LdBj$K$+$+$o$C$F$/$k$s$G$9$h$M!#>e5-$N$h$&$JM}M3$be(B
e$B$”$C$F!";d$Oe(BWindows-31Je$B$He(BShift_JISe$B$O$^$k$C$-$jF1$8!"$G$O$J$$e(B
e$BJ}$,$$$$$H;W$C$F$$$^$9!#e(B
e$B$b$C$H$b!"$8$c$“e(BISO-2022-JPe$B$N873J$J2r<a$O!)$H$+!”$$$m$$$me(B
e$BLdBj$,=P$F$/$k$H$O;W$$$^$9$,!D!D!#e(B

e$B$$$:$l$K$7$F$b!":G0->e5-!ZGX7J![$K=q$$$?$3$H$r2r7h$9$k$N$,e(B
e$BL\E*$G$9!#$=$l$,$G$-$k$N$J$i!“e(B String#valid_encoding? e$B0J30$Ne(B
e$B<jCJ$r;H$&$N$G$”$C$F$b9=$$$^$;$s!#e(B

e$B9b66@,5Ae(B ([email protected])

In article [email protected],
Masayoshi T. [email protected] writes:

e$B$$$C$?6q9g$G$9!#e(B
e$B$=$&$$$&5!G=$O$“$C$F$$$$$@$m$&$H$O;W$$$^$9$,!”$H$j$“$($:$A$g$Ce(B
e$B$H$7$?5?Ld$,$R$H$D$”$j$^$9!#e(B

UTF-8 e$B$b$=$&$$$&873J$J%A%'%C%/$r9T$&$s$G$9$+e(B?

e$B9b66@,5A$G$9!#e(B

Tanaka A. e$B$5$s$O=q$-$^$7$?e(B:

e$B$H$$$&$o$1$G!"Nc$($Pe(B String#valid_encoding? e$B$G873J$J%A%’%C%/$,e(B
e$B$G$-$k$H$&$l$7$$$G$9!#6qBNE*$K$O!"e(BShift_JISe$B$G!"e(BJISe$B$K$J$$J8;ze(B
e$B!J$$$o$f$k5!<o0MB8J8;z!K$,4^$^$l$F$$$k>l9g!"e(Bfalsee$B$K$J$k$He(B
e$B$$$C$?6q9g$G$9!#e(B

e$B$=$&$$$&5!G=$O$"$C$F$$$$$@$m$&$H$O;W$$$^$9$,!"$H$j$"$($:$A$g$Ce(B
e$B$H$7$?5?Ld$,$R$H$D$"$j$^$9!#e(B

e$B$R$H$D$@$1$G$9$+!)e(B

UTF-8 e$B$b$=$&$$$&873J$J%A%’%C%/$r9T$&$s$G$9$+e(B?

e$B<B:]$K$"$C$?MW7o$H$7$F$Oe(BUTF-8e$B$NHO0O$G$N873J$J%A%’%C%/$,I,MW!“e(B
e$B$H$$$&$3$H$O$”$j$^$;$s$G$7$?!#e(B
e$BF|K\8l0J30$G$NF~NO%A%’%C%/$K4X$7$F$O!“e(BASCIIe$B$K$D$$$F$Oe(B
e$B$$$o$f$ke(Bprintable asciie$B$N%A%’%C%/$r9T$C$?$j!”$5$i$Ke(B
e$B$=$N%5%V%;%C%H!J1Q?t;z$N$_$H$+!K$NF~NO@)8B$r9T$&$3$H$Oe(B
e$B$"$j$^$7$?$,!"$=$NB>$N8@8l$G$NF~NO@)8B$d!"$"$i$f$k8@8l$r$^$?$,$ke(B
e$BF~NO@)8B$H$$$&$N$O07$C$?$3$H$,$"$j$^$;$s!#e(B

UTF-8e$B$K4X$7$F!“e(BJIS X
0213e$B$N$$$o$f$kBhe(B3e$B!&Bhe(B4e$B?e=`$b4^$s$@!Ve(BJISe$B4A;z!W$G$Ne(B
e$BF~=PNO@)8B!”$H$$$&$N$b>-MhE*$K$O$"$j$&$k$H$O;W$C$F$$$^$9$,!"<B:]e(B
e$B$K$O$^$@$J$$>e!"7HBSBP1~$r9M$($k$H$7$P$i$/$O$J$$$+$J$H;W$$$^$9!#e(B

e$B0lJ}$G!"$=$N%"%W%j%1!<%7%g%s$G$N%f!<%6$+$i$NF~=PNO$,e(BUTF-8e$B$Ge(B
e$B$"$C$F$b!“FbItE*$K$$$o$f$ke(BJISe$B4A;zFb$+$I$&$+$N%A%’%C%/$,I,MWe(B
e$B$J>l9g$O$”$j$^$9e(B(e$B%9%H%l!<%8$d30It%7%9%F%`$H$NO"7H;~$K@)Ls$,e(B
e$B$"$k>l9g$J$I$G$9e(B)e$B!#$=$N>l9g$O!"e(BUTF-8e$B$N$^$^%A%’%C%/$G$-$J$/e(B
e$B$F$b!"B>$N%(%s%3!<%G%#%s%0$KJQ49$7$F$N%A%’%C%/$,9T$($l$Pe(B
e$B$h$$$H;W$$$^$9!#e(B

e$B9b66@,5Ae(B ([email protected])

e$B@.@%$G$9!#e(B

Masayoshi T. wrote:

e$B$G$-$k$H$&$l$7$$$G$9!#6qBNE*$K$O!"e(BShift_JISe$B$G!"e(BJISe$B$K$J$$J8;ze(B
e$B!J$$$o$f$k5!<o0MB8J8;z!K$,4^$^$l$F$$$k>l9g!"e(Bfalsee$B$K$J$k$He(B
e$B$$$C$?6q9g$G$9!#e(B

e$B$=$&$$$C$?%K!<%:$O$o$+$k$N$G$9$,!"$=$&$$$C$?e(B validation
e$B$KI,MW$J%G!<%?NLe(B
e$B$Oe(B Ruby e$BK\BN$,;}$D$K$OBg$-$9$.$k$h$&$K46$8$^$9!#e(BShift_JIS
e$B$G$9$ie(B
0x8140-0x9fEC, 0xE040-0xEAA4
e$B$^$G$-$C$A$j5M$^$C$F$$$k$+$H;W$$$-$d!“Hs4Ae(B
e$B;z$N$”$?$j$O$A$i$[$i7j$,$"$$$F$$$k$N$G!"$=$3$N>pJs$bI,MW$G$9!#e(BUnicode
e$B$@e(B
e$B$H$J$*$5$i!#e(B

e$B!ZHw9M![e(B
e$B$3$l$C$F!“e(BWindows-31Je$B$,e(BShift_JISe$B$Ne(Breplicae$B$=$N$^$^$G$$$$$N$+e(B
e$B$I$&$+LdBj$K$+$+$o$C$F$/$k$s$G$9$h$M!#>e5-$N$h$&$JM}M3$be(B
e$B$”$C$F!";d$Oe(BWindows-31Je$B$He(BShift_JISe$B$O$^$k$C$-$jF1$8!"$G$O$J$$e(B
e$BJ}$,$$$$$H;W$C$F$$$^$9!#e(B
e$B$b$C$H$b!"$8$c$“e(BISO-2022-JPe$B$N873J$J2r<a$O!)$H$+!”$$$m$$$me(B
e$BLdBj$,=P$F$/$k$H$O;W$$$^$9$,!D!D!#e(B

replica e$B$H$$$&8l46$,8m2r$r@8$_$d$9$$$N$+$b$7$l$^$;$s$,!"e(BRuby
e$BK\BN$,e(B (e$B$He(B
e$B$$$&$h$j$O54<V$,e(B)
e$B4X?4$r;}$C$F$$$k$N$OJ8;z$,$I$3$G@Z$l$k$+!"8@$$$+$($l$Pe(B
e$B$I$N$h$&$JId9f2=I=8=$rMQ$$$F$$$k$+$G$9!#$=$Ne(B encoding
(e$B$H$$$&$+e(B charset
e$B$H$$$&$+e(B e$BJ8;z=89ge(B)
e$B$K$*$$$F<B:]$KJ8;z$,Dj5A$5$l$F$$$k$+$I$&$+$O$I$&$G$be(B
e$B$$$$$N$G$9!#$=$7$F!"e(BShift_JIS e$B$be(B Windows-31J
e$B$b$I$A$i$b%7%U%HId9f2=I=8=e(B
e$B$rMQ$$$F$$$k0J>e$^$k$C$-$jF1$8$b$N$G$9!#e(BEUC-CN e$B$Oe(B EUC-KR
e$B$Ne(B replica e$B$G$9e(B
e$B$7$M!#e(B

e$B?^$K$9$k$He(B [ruby-dev:32952] e$B$_$?$$$J$+$s$8!#e(B

e$B$=$&$$$C$?@-<A$r;}$D0J>e!"8=:_$Ne(B valid_encoding?
e$B$KJ8;z=89g$X$NFbJq@-e(B
e$B%A%’%C%/$r5a$a$k$N$O0c$&$H;W$$$^$9!#$`$7$m!“BEEv@-$r%A%’%C%/$7$J$$$N$Ke(B
valid
e$B$rL>>h$k$N$,4V0c$C$F$$$k$N$G$O$J$$$+$H$$$&HcH=$O$”$j$($kOC$G!“e(B
well_formed_encoding?
e$B$H$7$?J}$,$$$$$N$G$O$J$$$+$HDs0F$7$h$&$H;W$C$?$b$Ne(B
e$B$N$”$^$j$NL>A0$ND9$5$K$d$a$?$N$G$9$,!"e(Bwell_formed?
e$B$J$iC;$$$N$G!"L>A0$re(B
e$BJQ$($^$;$s$+!#e(B

well-formed XML e$B$+$i$NO"A[$Ge(B validation e$B$r9T$o$:e(B Character
Encoding Form
(Scheme)
e$B$N$_$rH=Dj$9$k$N$@$m$&$H?dB,$7$F$/$l$k$3$H$,4|BT$G$-$^$9!#e(B

e$B$$$:$l$K$7$F$b!":G0->e5-!ZGX7J![$K=q$$$?$3$H$r2r7h$9$k$N$,e(B
e$BL\E*$G$9!#$=$l$,$G$-$k$N$J$i!“e(B String#valid_encoding? e$B0J30$Ne(B
e$B<jCJ$r;H$&$N$G$”$C$F$b9=$$$^$;$s!#e(B

e$B$H$3$m$G!“B>$K$I$N%3!<%I%]%$%s%H$,B8:$7!"$I$l$,B8:$7$J$$$+$rCN$C$F$$$ke(B
e$B$N$O!“J8;z%3!<%IJQ495!$”$?$j$,9M$($i$l$k$+$H;W$$$^$9!#B?$/$N%(%s%3!<%G%#e(B
e$B%s%0$G$O!”$=$l$,e(B Unicode
e$B$KJQ492DG=$+$GBEEv@-$r%A%’%C%/$G$-$k!D$+$N$h$&$Ke(B
e$B;W$($^$9$,!"$=$l$GK\Ev$K$$$$$N$G$9$+$M!#$^$:!"e(Bencoding
e$B$rCm0U?<$/A*$P$Je(B
e$B$$$H!"4|BT$7$?=89g$OF@$i$l$^$;$s!#e(BEUC-JP e$B$@$H$?$V$se(B JIS X
0212 e$B$,M>7W$Ge(B
e$B$9$7e(B (Perl/Encode e$B$@$He(B JIS X 0213
e$B$^$GF~$C$F$ke(B)e$B!“e(BShift_JIS e$B$G$be(B nkf e$B$de(B
e$B8=:_$Ne(B Ruby/transcode e$B$@$He(B CP932
e$B3HD%$bJQ49$7$^$9!#$=$b$=$b!”$$$o$f$kH>e(B
e$B3Q%+%J$dA43Q1Q?t;z$b4|BT$7$F$$$k=89g$KF~$C$F$$$J$$$s$8$c$J$$$G$9$+!#e(B

e$B7k6I$N$H$3$m!"$=$&$$$C$?e(B validation
e$B$O@lLg$N%b%8%e!<%k$KG$$;$k$Y$-$G!"HFe(B
e$BMQ$N2?$+$K5a$a$k$Y$-$G$O$J$$$H;W$$$^$9!#$G!"2?$r;H$($P$$$$$+$G$9$,!&!&!&e(B
Ruby/CHISE e$B$+$J!&!&!&!)e(B

e$B$=$l$b%"%l$J$s$G!"I,MW$J$i$Pe(B nkf
e$B$K$D$C$3$_$^$9$+$M$’!#!J$I$&$;FbIt$O6he(B
e$BE@%3!<%I$@$7!Ke(B

In article [email protected],
Masayoshi T. [email protected] writes:

e$B$R$H$D$@$1$G$9$+!)e(B

e$B$“$!!”$=$&$$$($P$b$&$R$H$D5?Ld$r;W$$$D$-$^$7$?!#e(B

Shift_JIS e$B$H$$$&$N$Oe(B IANA e$B$G=R$Y$i$l$F$$$kDj5A$G$9$+e(B?

e$B$D$^$j!“e(BJIS X0208:1997 e$B$Ne(B Appendix 1 e$B$K$”$ke(B JIS
X0201:1997,
JIS X0208:1997 e$B$NAH$_9g$o$;$G$9$+e(B?

e$B$=$l$H$b!"$=$N8e$KDj5A$5$l$?e(B Shift_JISX0213 e$B$H$+e(B
Shift_JIS-2004 e$B$G$7$g$&$+e(B?

e$B$“$k$$$Oe(B JIS X0208:1990, JIS X0208:1983, JIS X0208:1978
e$B$H$$$Ce(B
e$B$?$”$?$j$r;H$C$?$b$N$G$7$g$&$+e(B?

e$B<B:]$K$"$C$?MW7o$H$7$F$Oe(BUTF-8e$B$NHO0O$G$N873J$J%A%'%C%/$,I,MW!“e(B
e$B$H$$$&$3$H$O$”$j$^$;$s$G$7$?!#e(B

e$B$J$k$[$I!#e(B

e$B$^$D$b$He(B e$B$f$-$R$m$G$9e(B

In message “Re: [ruby-dev:33105] Re: String#valid_encoding? shoud be
strict?”
on Tue, 15 Jan 2008 03:48:40 +0900, Masayoshi T.
[email protected] writes:

|> UTF-8 e$B$b$=$&$$$&873J$J%A%‘%C%/$r9T$&$s$G$9$+e(B?
|
|e$B<B:]$K$"$C$?MW7o$H$7$F$Oe(BUTF-8e$B$NHO0O$G$N873J$J%A%’%C%/$,I,MW!“e(B
|e$B$H$$$&$3$H$O$”$j$^$;$s$G$7$?!#e(B
|e$BF|K\8l0J30$G$NF~NO%A%‘%C%/$K4X$7$F$O!"e(BASCIIe$B$K$D$$$F$Oe(B
|e$B$$$o$f$ke(Bprintable asciie$B$N%A%’%C%/$r9T$C$?$j!“$5$i$Ke(B
|e$B$=$N%5%V%;%C%H!J1Q?t;z$N$_$H$+!K$NF~NO@)8B$r9T$&$3$H$Oe(B
|e$B$”$j$^$7$?$,!“$=$NB>$N8@8l$G$NF~NO@)8B$d!”$“$i$f$k8@8l$r$^$?$,$ke(B
|e$BF~NO@)8B$H$$$&$N$O07$C$?$3$H$,$”$j$^$;$s!#e(B

e$B$=$l$O9b66$5$s$,F|K?M$@$+$i!)e(B

valid_encoding?e$B$H$$$&$N$Oe(BCESe$BE*$Ke(Bvalide$B$+$I$&$+$rH=Dj$9$k$s$@e(B
e$B$H;W$$$^$9!#$G!“9b66$5$s$NM_$7$$!V873J$J%A%'%C%/!W$H$$$&$N$Oe(B
e$B$=$NGX8e$K$”$kJ8;z=89g$rLd$&$F$$$k$N$G$O$J$$$+$H!#e(B

e$B$G!"$=$l$Oe(BAPIe$B$H$7$FJL$K$7$?J}$,$h$$$H;W$$$^$9!#;H$$J}$b<BAu$Ne(B
e$B;EAH$_$b0[$J$k$N$G!#e(B

e$B$I$A$i$+$H$$$&$HJQ497O$G$7$g$&$M!#e(BMartin
e$B$5$s$b$*$C$7$c$C$F$$e(B
e$B$^$7$?e(B[ruby-core:15046]e$B$,!"e(Bencodee$B$KBh;00z?t$d%V%m%C%/$r;XDj$9e(B
e$B$k$3$H$Ge(B

e$B!J$$5$^$C$F$$$J$$>l9g$O:FF~NO$r$&$J$,$7$?$j!"!Ve(B?e$B!W$K6/@)E$Ke(B
e$BCV49$7$?$j$7$^$9!K!#e(B

e$B$,$G$-$k$h$&$K$J$k$N$G$O$J$$$+$H;W$$$^$9!#e(B

                            e$B$^$D$b$He(B e$B$f$-$R$me(B /:|)

At 02:24 08/01/15, Tanaka A. wrote:

e$B$H$$$&$o$1$G!"Nc$($Pe(B String#valid_encoding? e$B$G873J$J%A%'%C%/$,e(B
e$B$G$-$k$H$&$l$7$$$G$9!#6qBNE*$K$O!"e(BShift_JISe$B$G!"e(BJISe$B$K$J$$J8;ze(B
e$B!J$$$o$f$k5!<o0MB8J8;z!K$,4^$^$l$F$$$k>l9g!"e(Bfalsee$B$K$J$k$He(B
e$B$$$C$?6q9g$G$9!#e(B

e$B$=$&$$$&5!G=$O$“$C$F$$$$$@$m$&$H$O;W$$$^$9$,!”$H$j$“$($:$A$g$Ce(B
e$B$H$7$?5?Ld$,$R$H$D$”$j$^$9!#e(B

UTF-8 e$B$b$=$&$$$&873J$J%A%'%C%/$r9T$&$s$G$9$+e(B?

e$B54<V$NJ8;zB0@-%F%9%He(B (\p{Hiragana}e$B$H$+e(B)
e$B$r$&$^$$6q9gAH$9g$o$;$l$Pe(B
e$B8=:4{$K2DG=$@$H;W$$$^$9!#$7$+$7%P!<%8%g%s0MB8$G$9!#e(B
e$BB>$NJ8;z%3!<%I$G$OJQ49$7$F$
$k$N$O3N$+$K0l$D$N<j$G$9!#0U$^$G$be(B
e$B!VJQ49$7$F$
$F!"Nc30$,H/@8$7$?$i%A%'%C%/<:GT!W$G2DG=$G$9!#e(B

e$B59$7$/$*4j$$$7$^$9!#e(B Martin.

#-#-# Martin J. Du"rst, Assoc. Professor, Aoyama Gakuin University
#-#-# http://www.sw.it.aoyama.ac.jp mailto:[email protected]

e$B@.@%$G$9!#e(B

Martin D. wrote:

e$B!ZDs0F![e(B
e$B8=:4{$K2DG=$@$H;W$$$^$9!#$7$+$7%P!<%8%g%s0MB8$G$9!#e(B
e$BB>$NJ8;z%3!<%I$G$OJQ49$7$F$
$k$N$O3N$+$K0l$D$N<j$G$9!#0U$^$G$be(B
e$B!VJQ49$7$F$_$F!"Nc30$,H/@8$7$?$i%A%'%C%/<:GT!W$G2DG=$G$9!#e(B

Unicode Scripts e$B$O3N$+$KM-MQ$J$N$G$9$,!":#2s4|BT$5$l$F$$$k=89g$Oe(B
JIS X
0208 (e$B$^$?$O$=$l$K=$@5$r2C$($?$b$Ne(B) e$B$@$H$*$b$&$N$G!"e(B\p{Han}
e$B$K$J$C$F$7$^e(B
e$B$&e(B Unicode Scripts e$B$G$OFq$7$$$H;W$$$^$9!#e(B

e$B4|BT$7$F$$$k=89g$O$?$V$s$3$N$X$se(B

N3091
e$B$,F~$l$P$$$$$N$G$7$g$&$,!“$O$?$7$F?JE8$,$”$k$N$+$OCN$j$^$;$s!#e(B
http://www.ucam.ac.ma/fssm/rydarab/doc/document/UnicodePropositions.htm

e$B$H$_$?$G$9!#e(B

On Tue, 15 Jan 2008 01:59:42 +0900
Masayoshi T. [email protected] wrote:

e$B!ZGX7J![e(B
e$BF~NO%A%'%C%/Ey$G!“$”$kJ8;zNs$,$“$k%(%s%3!<%G%#%s%0$NHO0OFb$Ke(B
e$B$$5$^$C$F$$$k$+!“873J$K%A%'%C%/$7$?$$>l9g$,$”$j$^$9!#e(B
e$B!J$
$5$^$C$F$$$J$$>l9g$O:FF~NO$r$&$J$,$7$?$j!”!Ve(B?e$B!W$K6/@)E*$Ke(B
e$BCV49$7$?$j$7$^$9!K!#e(B
e$B!A!Ae(B
e$B$$$:$l$K$7$F$b!":G0->e5-!ZGX7J![$K=q$$$?$3$H$r2r7h$9$k$N$,e(B
e$BL\E*$G$9!#$=$l$,$G$-$k$N$J$i!“e(B String#valid_encoding? e$B0J30$Ne(B
e$B<jCJ$r;H$&$N$G$”$C$F$b9=$$$^$;$s!#e(B

1.9 e$B$G$O$J$/$Fe(B 1.8.6
e$B$G;n$7$?7k2L$G$9$7!"=hM}7O$K0MB8$9$k$h$&$J5$$b$9$ke(B
e$B$N$G$9$,!"e(BIconv e$B$r;H$&$N$O$I$&$G$7$g$&$+!)e(B

$ ruby -riconv -e ‘Iconv.iconv(“CP932”,“CP932”,“\x87\x40”)’

$ ruby -riconv -e ‘Iconv.iconv(“SJIS”,“SJIS”,“\x87\x40”)’
-e:1:in `iconv’: “\207@” (Iconv::IllegalSequence)
from -e:1

e$BNc30$r=&$C$FE,Ev$K=hM}$9$l$P!"!Ve(B?e$B!W$KJQ49$9$k$3$H$b$G$-$k$H;W$$$^$9!#e(B