Converting file from utf-16 to utf-8

Hi,

I’m currently rewriting an old php-script in ruby.
One thing this script is for is to open a file which is in utf-16, read
it as utf-8, work a little bit with the data and save it into a
database.

However I tried to use Iconv to solve this but I’ve failed :wink:

The php script runs perfectly without any error.
If somebody is familiar with php here is the code (yeah, very VERY ugly,
but that’s why I’m rewriting it :))

$fp = fopen($file,“r”);
$lines = file($file);
for ($line=1; $line<=count($lines); $line++) {
$string .= fgets($fp);
}
$string = iconv(‘UTF-16’, ‘UTF-8’, $string);

But Ruby throws an Iconv::IllegalSequence error:

doc = File.open(file).read
doc = Iconv.iconv(‘utf-16’, ‘utf-8’, doc) #Iconv::IllegalSequence

Because it’s the same file I really don’t know how to solve this.
If I’m using utf-16//IGNORE, obviously, many signs are missing.
Are any other “workarounds”/methods to use out there?
Or something I overlooked?

sincerely.

OZAWA Sakuro wrote:

Iconv.iconv takes “to” encoding first, followed by “from” encoding.

Oh man… that’s really… uhm… head->wall
Read the documentation many times and didn’t notice it.

I’m really sorry to have opened such a thread T__T
But thanks for the answer and not flaming about it :wink:

On 3/23/2010 9:17 AM, Kioko – wrote:

Heh, I’m sure everyone here has never, ever, ever made a mistake
like that.

On Wed, Mar 24, 2010 at 00:07, Kioko – [email protected] wrote:

$string = iconv(‘UTF-16’, ‘UTF-8’, $string);

doc = Iconv.iconv(‘utf-16’, ‘utf-8’, doc) #Iconv::IllegalSequence

Iconv.iconv takes “to” encoding first, followed by “from” encoding.


OZAWA Sakuro

“The future will be better tomorrow.” --Dan Quayle