[Redirected to ruby-core so that James can also read this.]
Hello James,
This is an error report from Yugui about a csv test
failing on a Mac.
The reason for the failure is line 498 in lib/csv.rb,
in method CSV#inspect. This line reads:
str.map { |s| s.encode(“ASCII-8BIT”) }.join
The reason for the failure is that currently, filenames on a Mac
are labeled as being in an “encoding” of UTF8-MAC. The label
UTF8-MAC is used to mark the assumption that this string is in a
character normalization form particular to the Mac (mostly NFD,
but not for Korean, and not for CJKV compatibility ideographs,
as far as I understand).
There is in general no knowledge about character normalization with
respect to strings labeled UTF-8 (and even for UTF8-MAC, there is
no guarantee about character normalization at all). In my personal
view, the value of UTF8-MAC is questionable at least at the current
point in time where we do not handle character normalization in
any particular way. But for the current bug, that’s actually a side
issue. We might be able to fix this by introducing a (dummy) conversion
from UTF8-MAC to UTF-8, but that won’t actually fix the real problem.
The real problem is that the line above ignores that conversion
to ASCII-8BIT only works for US-ASCII characters, but not for
all the other characters that might appear e.g. in a filename.
The easiest fix for this, which is probably what was intended,
is to change the above line to
str.map { |s| s.force_encoding(“ASCII-8BIT”) }.join
A slightly more “user-friendly” fix is to change this to
something like:
begin
str.join
rescue
str.map { |s| s.force_encoding(“ASCII-8BIT”) }.join
end
This will only do a force_encoding if the encodings can’t
be joined as is.
[The code above hasn’t been tested; I don’t have access to a Mac.]
Hope this helps. Regards, Martin.
At 12:27 08/12/25, Yugui (Yuki S.) wrote:
nil
test_inspect_is_smart_about_io_types’
/Users/yugui/src/ruby/mri/test/csv/test_features.rb:233:in
`test_inspect_is_smart_about_io_types’
134 tests, 1886 assertions, 0 failures, 1 errors, 0 skips
make: *** [test-all] Error 1
–
Yugui [email protected]
http://yugui.jp
e$B;d$O;d$re(BDumpe$B$9$ke(B
#-#-# Martin J. Du"rst, Assoc. Professor, Aoyama Gakuin University
#-#-# http://www.sw.it.aoyama.ac.jp mailto:[email protected]