Same character show different code in two machine

Ruby
1

one chinese character show different code in two different machine.

machine A: \243\244
machine B: \302\245

so I have to using different pattern for two machines, like this:
machine A: text.split("\243\244")
machine B: text.split("\302\245")

I know this is not the proper way, but could some one tell me:
what is the root course?
What different between machine A and B?
what is the proper way to handle this ?

thanks very much!

-ryan

2

Ryan S. wrote:

one chinese character show different code in two different machine.

machine A: \243\244
machine B: \302\245

so I have to using different pattern for two machines, like this:
machine A: text.split("\243\244")
machine B: text.split("\302\245")

I know this is not the proper way, but could some one tell me:
what is the root course?
What different between machine A and B?
what is the proper way to handle this ?

thanks very much!

-ryan

Are the two machines getting the text from the same source?
What is the source?
What is the encoding of the source?
What locale is Ruby running under?
What version of Ruby are you running?
What encoding is the relevant Ruby source file set to?
How are you retrieving the text from the source?
What encoding is the string once its finally retrieved?

3

Are the two machines getting the text from the same source?
yes, from a same public web page. like http://www.abc.com/xyz.htm

What is the source?
as above, source is webpage, using Watir to parse the webpage.

What is the encoding of the source?
text/html; charset=utf-8

What locale is Ruby running under?
you mean Windows system locale?
Machine A is “Win XP Pro (English version)”, system locale is “PRC”
Machine B is “Win XP Pro (Chinese version)”, system locale is “中文(中国)”
So in my point, A and B has indentical system locale.

What version of Ruby are you running?
Machine A and B both are
ruby 1.8.6 (2008-08-11 patchlevel 287) [i386-mswin32]

What encoding is the relevant Ruby source file set to?
A and B using same set of ruby script, their are in utf-8 encoding. both
has this line in the head of script.

-- coding: utf-8 --

How are you retrieving the text from the source?
Using Watir to parse text in the webpage.

What encoding is the string once its finally retrieved?
I have no idea with this, How to get this?

4

Thanks, Walton,

need include something?

irb(main):006:0> “Hello”.encoding
NoMethodError: undefined method `encoding’ for “Hello”:String
from (irb):6

5

On 2/7/2010 5:51 PM, Ryan S. wrote:

What encoding is the string once its finally retrieved?
I have no idea with this, How to get this?

irb(main):001:0> “Hello”.encoding
=> #Encoding:IBM437

Walton

6

Ryan S. wrote:

one chinese character show different code in two different machine.

machine A: \243\244
machine B: \302\245

In hex those are: \xa3\xa4
\xc2\xa5

The first is not valid UTF-8. I suppose it might be UTF-16: U+A3A4 or
U+A4A3 depending on little or big-endian. Or it could be some older
proprietary Asian encoding.

The second of these could be UTF-8. If so it would be codepoint 165, the
‘yen’ symbol. Or it could be U+C2A5 in UTF-16.

7

Thanks Brian, see my in line comments.

The first is not valid UTF-8. I suppose it might be UTF-16: U+A3A4 or
U+A4A3 depending on little or big-endian. Or it could be some older
proprietary Asian encoding.

[Ryan] How to correct this (to UTF-8), it is a English XP Pro with PRC
as system locale.

The second of these could be UTF-8. If so it would be codepoint 165, the
‘yen’ symbol. Or it could be U+C2A5 in UTF-16.

[Ryan] yes, it is chinese currency CNY ‘yen’ symbol.

8

Ryan S. wrote:

The first is not valid UTF-8. I suppose it might be UTF-16: U+A3A4 or
U+A4A3 depending on little or big-endian. Or it could be some older
proprietary Asian encoding.

[Ryan] How to correct this (to UTF-8), it is a English XP Pro with PRC
as system locale.

Sorry, I have no idea. Are you sure that \xa3\xa4 correponds exactly to
that one character? Is the rest of the encoding variable length or fixed
length? (e.g. are all characters two bytes long, even a western letter
“A”?)

Questions about Microsoft operating systems and what encodings they use
really belong in a Microsoft users’ forum, as it’s not anything to do
with Ruby.

9

Ryan S. wrote:

Thanks, Walton,

need include something?

irb(main):006:0> “Hello”.encoding
NoMethodError: undefined method `encoding’ for “Hello”:String
from (irb):6

No, I don’t think that method exists in 1.8.

Best,
–Â
Marnen Laibow-Koser
http://www.marnen.org
[email protected]

10

Brian C. wrote:

Ryan S. wrote:

The first is not valid UTF-8. I suppose it might be UTF-16: U+A3A4 or
U+A4A3 depending on little or big-endian. Or it could be some older
proprietary Asian encoding.

[Ryan] How to correct this (to UTF-8), it is a English XP Pro with PRC
as system locale.

Sorry, I have no idea. Are you sure that \xa3\xa4 correponds exactly to
that one character? Is the rest of the encoding variable length or fixed
length? (e.g. are all characters two bytes long, even a western letter
“A”?)

Questions about Microsoft operating systems and what encodings they use
really belong in a Microsoft users’ forum, as it’s not anything to do
with Ruby.

I have no idea either, but I will upgrade to ruby 1.9 to leverage
string.encoding feature. thank you.