# Saving the web, charset problems and symbols problems **URL:** \ **Category:** Ruby\ **Created:** [January 30, 2009, 6:08am UTC](https://www.ruby-forum.com/t/saving-the-web-charset-problems-and-symbols-problems/158996 "2009-01-30T06:08:20Z")\ **Posts on this page:** 1\ **Page:** 1
**Author:** ![saknarede](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/f475e1/32.png) [@saknarede](https://www.ruby-forum.com/u/saknarede)\ **Post date:** [January 30, 2009, 6:08am UTC](https://www.ruby-forum.com/t/saving-the-web-charset-problems-and-symbols-problems/158996/1 "2009-01-30T06:08:20Z")
Hi all! I think that a lot of ruby scripts are for web crawling, web scrapping and many more applications with the web. I’m working with the web too, I try to save text of many different webs. In this moment I’m trying to solve two problems: 1 - How to standard the charset of the web. There are a lot of differents charsets and I think that it must be possible another solution that see every charset and convert to proper charset each time. (By the way, what is the best method to see charset of a file? command file is not very good, I think) 2 - How to convert HTML to plain text. I use Hpricot but a lot of very rare simbols continues there like “€” or “””. Wich is the most used method? Thanks a lot