Code Review: EMS10

tfpt review “/shelveset:EMS10;REDMOND\tomat”
Comment :
Implements support for extension methods in IronRuby.

DLR:

  • Factors out reusable parts of type inferer to ReflectionUtils and
    simplifies the inferer a bit. We need to bind generic parameters when
    dealing with extension methods whose first parameter is an open
    constructed type.
  • Moves around a few methods in ReflectionUtils so they are better
    organized to regions.
  • Implements extension method reflection services in ReflectionUtils.
    The implementation will improve when we’ll get our new metadata reader
    checked-in. We’ll also need a slightly different extension method
    enumeration scheme for Python. I’ll also rename ReflectionUtils to
    ReflectionServices when they use the new metadata reader and move it out
    of Utils directory. They are becoming more than just simple utils.
  • Updates OverloadResolver to prefer non-extension methods over
    extension methods if it can’t decide which overload is better based on
    other criteria.
  • Implements a workaround for a bug in MethodInfo.GetBaseMethod in
    CompilerHelpers.TryGetCallableMethod. The API might return a generic
    method definition even if called on a method instantiation.
  • Small improvements to a couple of ArrayUtils.

Ruby:

  • Adds method Kernel#using_clr_extensions that takes a namespace and
    activates all extension methods defined on classes defined in that
    namespace (doesn’t include sub-namespaces) regardless of the assembly
    they are defined in. All assemblies already loaded to the current
    ScriptRuntime are reflected for extension methods and any assembly that
    is loaded in the future gets reflected as well. This means that
    additional extension methods might appear when a new assembly is loaded.
    For each affected type the loaded extension methods are added to the
    overload sets of existing methods (if there are any). Thus if class C
    defines an instance method foo() and an extension method foo(this C,
    int) is loaded the next call of foo on an instance of C will trigger
    overload resolution algorithm to choose among overloads { foo() and
    foo(int) }. If the resolver can’t chose based upon the call-site
    arguments the non-extension methods are preferred.
  • Generic extension methods might be defined, for example like so:

public static void Foo<S, T>(this Dictionary<S, T> x, T y) where S : T
{ }

This defined Foo on all Dictionaries whose key type is the same or
derives from the type of the values stored in the dictionary. This is
supported to the extent that all involved constraints can be
instantiated using the type of the first argument to the call-site. We
do ignore constraints that can’t. For example:

public static void Foo<S, T>(this S x, T y) where S : T { }

In this case the methods will be available on all types in the system.
However if the method is invoked with an argument that is not compatible
with the constraint the overload can’t be selected and the method call
might fail.

If the first parameter of an extension method is not a constructed
type (i.e. it is a plain generic parameter like in the example above)
the method is supposed to be defined on every type in the system that
satisfies the constrains of the generic parameter. In the current
implementation IronRuby considers such method (say Foo) defined on
Object and available to all applicable types via inheritance. Thus a
Ruby method Foo or foo defined on some class A <: Object hides the
extension method Foo. This means that the extension method won’t be
available from any subclass of A until the Ruby method on A gets
removed. Ideally the extension method should appear to be defined
directly on every type instead of just Object. However this brings
additional complexity to the current IronRuby type system. Since
defining an extension method on every single type is rather a corner
case I’ve decided for the simple approach.

  • We intentionally don’t enable extension methods on built-in types
    (such as Array or Hash) for two reasons:
    o There is high potential for name conflicts. For example,
    IEnumerable LINQ extensions include methods like First, Zip, Count,
    Max, Min, Sum, etc. some of these are already methods on Ruby’s
    Enumerable and other might easily be in future. We could use the policy
    of non-mangling names so that they could be called only using their CLR
    name and not the Ruby equivalent (Zip vs zip). However, then writing
    LINQ queries that work for all data sources would require to always use
    CLR names otherwise it wouldn’t work for builtins.
    o Built-ins hide CLR interface they implement to be fully compatible
    with Ruby built-ins. For example, although Array implements
    IList Array.included_mixins == [Enumerable]. Thus it would need
    a bit of magical behavior to make the extension methods appear on Array.
    It would actually be much cleaner if a separate non-built-in class was
    defined that wraps Ruby array and implements IEnumerable. Then any
    time you want to use an array as LINQ data source you just wrap it into
    this enumerable implementation.

  • An example of using LINQ extension methods:

load_assembly “System.Core”
using_clr_extensions System::Linq

p System::String.to_clr_type.get_methods.
where(System::Func[Object, System::Boolean].new { |m| m.name[0, 2]
== ‘To’ }).
select(System::Func[Object, Object].new { |m| m.name }).
to_a

=> [‘ToCharArray’, ‘ToCharArray’, ‘ToLower’, ‘ToLower’,

‘ToLowerInvariant’, ‘ToUpper’, ‘ToUpper’, ‘ToUpperInvariant’,
‘ToString’, ‘ToString’]

Currently IronRuby’s type inference doesn’t infer the generic
parameters of Where and Select methods so we need to help it by
specifying the types explicitly.

  • Fixes name mangling of “or” word.
  • Fixes cross-runtime class hierarchy locking bug in constant
    resolution. It is no longer allowed to open a foreign module or class
    via module/class keyword.

Tomas