Code Review: EncodingX6

tfpt review “/shelveset:EncodingX6;REDMOND\tomat”

Adds RubyCompatibility enum that specifies the major version of Ruby
that IronRuby should be compatible with.
Adds -18, -19 and -20 command line options that switch the
compatibility mode to Ruby 1.8, 1.9 and 2.0 respectively.
For now the default compatibility mode is 1.8. The difference between
-18 and -19 is currently mostly related to Unicode and string encodings.
The encodings support is still quite experimental therefore -19 is not a
default mode.

Makes RubyEncoding instances per-domain singletons.
Refactors tokenizer and implements \uXXXX and \u{code-point} string
literal escapes.

Makes StringLiteral a separate node (Literal was previously used for
strings as well).
Adds StringLiteralEncoding which defines what kind of characters are
contained in the literal.
Mixing encoded characters, \u and binary escapes (\xXX, ###) doesn’t
work yet.

Replaces char? with int in libraries and fixes character handling in
some library methods.
Adds external_encoding, internal_encoding to IO class, although only
external_encoding is used so far.

Tomas