# Zlib gzip Iconv, what is going on with UTF-8

**URL:** <https://www.ruby-forum.com/t/zlib-gzip-iconv-what-is-going-on-with-utf-8/182691>\
**Category:** Ruby\
**Created:** [February 1, 2010, 3:13pm UTC](https://www.ruby-forum.com/t/zlib-gzip-iconv-what-is-going-on-with-utf-8/182691 "2010-02-01T15:13:46Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![inprogs](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/8491ac/32.png) [@inprogs](https://www.ruby-forum.com/u/inprogs)\
**Post date:** [February 1, 2010, 3:13pm UTC](https://www.ruby-forum.com/t/zlib-gzip-iconv-what-is-going-on-with-utf-8/182691/1 "2010-02-01T15:13:46Z")

</div>

Hi. I googled rly hard but nothing is working.

I got ruby 1.8.6 (test has been performed on windows)

My problem is when I try receive gziped response from some sites which  
is not in UTF-8 itself.

For instance this one is correct:  
response = Net::HTTP.get\_with\_head(‘[http://www.wp.pl/](http://www.wp.pl/)’,  
{‘Accept-Encoding’ =\> ‘gzip;q=1.0, identity;’, ‘Accept-Charset’ =\>  
‘utf-8’})

so when I try unpack it everything works fine  
if response[‘Content-Encoding’]  
body\_io = StringIO.new(response.body)  
html = Zlib::GzipReader.new(body\_io).read()  
html = Iconv.conv(‘utf-8//IGNORE’, encoding, html)  
else  
#html = response.body  
end

problem APPERAS when site which I want to receive has got other charset  
than UTF-8, so changing first line to(other server):  
response = Net::HTTP.get\_with\_head(‘[http://www.interia.pl/](http://www.interia.pl/)’,  
{‘Accept-Encoding’ =\> ‘gzip;q=1.0, identity;’, ‘Accept-Charset’ =\>  
‘utf-8’})

give me site without ANY PROPER UTF-8 character

when I comment out Iconv line

my output has got abnormal characters in the utf-8 character’s place  
like: “?” (every character is replaced with some sort of question mark)

It is fault of server Gzip-way is content packed or my fault (I unpack  
it in bad way)

Regards

---

<div class="post-metadata">

**Author:** ![inprogs](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/8491ac/32.png) [@inprogs](https://www.ruby-forum.com/u/inprogs)\
**Post date:** [February 1, 2010, 4:00pm UTC](https://www.ruby-forum.com/t/zlib-gzip-iconv-what-is-going-on-with-utf-8/182691/2 "2010-02-01T16:00:14Z")

</div>

I want to add, everything works fine when I turn off gzip  
so following code works fine for every site:

response = Net::HTTP.get\_with\_head(‘[http://www.interia.pl/](http://www.interia.pl/)’,  
{‘Accept-Charset’ =\> ‘utf-8’})

that = Nokogiri::HTML(Iconv.conv(‘utf-8//IGNORE’, encoding, html))

and ‘that’ is proper formated UTF-8 characterset

When gzip is present makes it ���

regards

---

<div class="post-metadata">

**Author:** ![inprogs](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/8491ac/32.png) [@inprogs](https://www.ruby-forum.com/u/inprogs)\
**Post date:** [February 1, 2010, 5:08pm UTC](https://www.ruby-forum.com/t/zlib-gzip-iconv-what-is-going-on-with-utf-8/182691/3 "2010-02-01T17:08:25Z")

</div>

Piotr MÄ…sior wrote:

> I want to add, everything works fine when I turn off gzip  
> so following code works fine for every site:
> 
> response = Net::HTTP.get\_with\_head(‘[http://www.interia.pl/](http://www.interia.pl/)’,  
> {‘Accept-Charset’ =\> ‘utf-8’})
> 
> that = Nokogiri::HTML(Iconv.conv(‘utf-8//IGNORE’, encoding, html))
> 
> and ‘that’ is proper formated UTF-8 characterset
> 
> When gzip is present makes it ���
> 
> regards

Problem SOLVED, I had bad condition what caused problem. I always was  
giving “utf-8” to Iconv as encoding

regards