# Mechanize and encoding

**URL:** <https://www.ruby-forum.com/t/mechanize-and-encoding/153916>\
**Category:** Ruby\
**Created:** [November 22, 2008, 11:33pm UTC](https://www.ruby-forum.com/t/mechanize-and-encoding/153916 "2008-11-22T23:33:04Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![aspartam](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/ebca7d/32.png) [@aspartam](https://www.ruby-forum.com/u/aspartam)\
**Post date:** [November 22, 2008, 11:33pm UTC](https://www.ruby-forum.com/t/mechanize-and-encoding/153916/1 "2008-11-22T23:33:04Z")

</div>

I’m trying to scrape a page that both HTTP-header and the HMTL document  
claim is UTF-8, but all special characters are substituted by a question  
mark when I use Mechanize/Hpricot to scrape some accented strings and  
save to a local file. I suspect the page is in “ISO-8859-1”, but I’m not  
sure.

I have tried using the"ruby -Ku" and also the $KCODE=‘u’ option without  
success.

How can I force Mechanize to read the doc as “ISO-8859-1”?

I understand that Iconv can convert encoding, but just can’t see how I  
can use it with Mechanize…

Thanks,  
Marius

---

<div class="post-metadata">

**Author:** ![aspartam](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/ebca7d/32.png) [@aspartam](https://www.ruby-forum.com/u/aspartam)\
**Post date:** [December 2, 2008, 4:03pm UTC](https://www.ruby-forum.com/t/mechanize-and-encoding/153916/2 "2008-12-02T16:03:34Z")

</div>

I have had exactly the same problem and the same question.

It seems I solve it with $KCODE =‘UTF8’.