Forum: Ruby Mechanize and encoding

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
Marius H. (Guest)
on 2008-11-23 00:33
I'm trying to scrape a page that both HTTP-header and the HMTL document
claim is UTF-8, but all special characters are substituted by a question
mark when I use Mechanize/Hpricot to scrape some accented strings and
save to a local file. I suspect the page is in "ISO-8859-1", but I'm not
sure.

I have tried using the"ruby -Ku" and also the $KCODE='u' option without
success.

How can I force Mechanize to read the doc as "ISO-8859-1"?

I understand that Iconv can convert encoding, but just can't see how I
can use it with Mechanize...

Thanks,
Marius
عمر Ù. (Guest)
on 2008-12-02 17:03
I have had exactly the same problem and the same question.

It seems I solve it with $KCODE ='UTF8'.
This topic is locked and can not be replied to.