Forum: Ruby-core Refinements and nested methods

Issue #4085 has been updated by shugo (Shugo Maeda).

Assignee changed from shugo (Shugo Maeda) to matz (Yukihiro Matsumoto)

Thanks for your feedback, Charles and others.
I understand your worries.

The feature set of Ruby 2.0 has already been frozen, so it's impossible 
to introduce a completely different feature in Ruby 2.0.  So we have 
only the following options:

1. introduce the whole features of Refinements currently implemented
2. introduce some of the features of Refinements (= drop some features)
3. remove all features of Refinements

I think optional features of Refinements are as follows:

A. refinement inheritance in class hierarchies
B. refinement activation for reopened module definitions
C. refinement activation for the string version of 
module_eval/instance_eval
D. refinement activation for the block version of 
module_eval/instance_eval

I've asked Matz to decide whether Refinements should be included in Ruby 
2.0, and if so, which of these features should be included.

My own take is as follows:

* I'm not sure A is good or not.  It's useful in some cases, but it may 
be confusing because include doesn't inherit refinements, but class 
inheritance does.  So it's OK to remove A from Ruby 2.0.
* I want C and D for internal DSLs, but D might be difficult to 
implement in VMs other than CRuby.  So it's OK to remove D from Ruby 
2.0.
  FYI, I'm implementing it without performance overhead when refinements 
are not used.
    http://shugo.net/tmp/refinement_fix_1119.diff
  In this implementation, refined methods are stored in neither an 
inline method cache nor the global method cache,
  so there's no need to invalidate cache for module_eval.  I hope D will 
be introduced in the future.
* From the perspective of consistency, C and D depend on B.  So if C or 
D is included in Ruby 2.0, B should also be included.

And, I explain some things to clarify my intention.

I have used the word "lexical" to describe Refinements, but by the word 
I've meant just that Refinements doesn't support local rebinding.  For 
example, in the following code, FooExt doesn't affect Bar#call_foo even 
if it's called from Baz, which is a module using FooExt.

  class Foo
  end
  module FooExt
    refine Foo do
      def foo
        puts "foo"
      end
    end
  end
  class Bar
    def call_foo(f)
      f.foo
    end
  end
  module Baz
    using FooExt
    f = Foo.new
    f.foo                # => foo
    Bar.new.call_foo(f)  # => NoMethodError
  end

I think it's the most important feature of Refinements.  Without it, 
it's hard to avoid conflicts among multiple refinements.

Some people seem to suspect that code using refinements is difficult to 
debug, but reflection APIs may be useful to debug such code.

  module M
    refine Fixnum do
      def foo; puts "foo" end
    end
  end
  using M
  p 123.method(:foo).owner #=> #<refinement:Fixnum@M>

I admit that Refinements are complex, but it's because issues to address 
by Refinements are themselves complex.  And, I think Refinements should 
not be over-used.  Application programmers should not use Refinements. 
Refinements are for library/framework programmers.  Besides, even if 
you're a library or framework programmer, consider other features such 
as subclassing before Refinements.

----------------------------------------
Feature #4085: Refinements and nested methods
https://bugs.ruby-lang.org/issues/4085#change-33146

Author: shugo (Shugo Maeda)
Status: Assigned
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category: core
Target version: 2.0.0


=begin
 As I said at RubyConf 2010, I'd like to propose a new features called
 "Refinements."

 Refinements are similar to Classboxes.  However, Refinements doesn't
 support local rebinding as mentioned later.  In this sense,
 Refinements might be more similar to selector namespaces, but I'm not
 sure because I have never seen any implementation of selector
 namespaces.

 In Refinements, a Ruby module is used as a namespace (or classbox) for
 class extensions.  Such class extensions are called refinements.  For
 example, the following module refines Fixnum.

   module MathN
     refine Fixnum do
       def /(other) quo(other) end
     end
   end

 Module#refine(klass) takes one argument, which is a class to be
 extended.  Module#refine also takes a block, where additional or
 overriding methods of klass can be defined.  In this example, MathN
 refines Fixnum so that 1 / 2 returns a rational number (1/2) instead
 of an integer 0.

 This refinement can be enabled by the method using.

   class Foo
     using MathN

     def foo
       p 1 / 2
     end
   end

   f = Foo.new
   f.foo #=> (1/2)
   p 1 / 2

 In this example, the refinement in MathN is enabled in the definition
 of Foo.  The effective scope of the refinement is the innermost class,
 module, or method where using is called; however the refinement is not
 enabled before the call of using.  If there is no such class, module,
 or method, then the effective scope is the file where using is called.
 Note that refinements are pseudo-lexically scoped.  For example,
 foo.baz prints not "FooExt#bar" but "Foo#bar" in the following code:

   class Foo
     def bar
       puts "Foo#bar"
     end

     def baz
       bar
     end
   end

   module FooExt
     refine Foo do
       def bar
         puts "FooExt#bar"
       end
     end
   end

   module Quux
     using FooExt

     foo = Foo.new
     foo.bar  # => FooExt#bar
     foo.baz  # => Foo#bar
   end

 Refinements are also enabled in reopened definitions of classes using
 refinements and definitions of their subclasses, so they are
 *pseudo*-lexically scoped.

   class Foo
     using MathN
   end

   class Foo
     # MathN is enabled in a reopened definition.
     p 1 / 2  #=> (1/2)
   end

   class Bar < Foo
     # MathN is enabled in a subclass definition.
     p 1 / 2  #=> (1/2)
   end

 If a module or class is using refinements, they are enabled in
 module_eval, class_eval, and instance_eval if the receiver is the
 class or module, or an instance of the class.

   module A
     using MathN
   end
   class B
     using MathN
   end
   MathN.module_eval do
     p 1 / 2  #=> (1/2)
   end
   A.module_eval do
     p 1 / 2  #=> (1/2)
   end
   B.class_eval do
     p 1 / 2  #=> (1/2)
   end
   B.new.instance_eval do
     p 1 / 2  #=> (1/2)
   end

 Besides refinements, I'd like to propose new behavior of nested 
methods.
 Currently, the scope of a nested method is not closed in the outer 
method.

   def foo
     def bar
       puts "bar"
     end
     bar
   end
   foo  #=> bar
   bar  #=> bar

 In Ruby, there are no functions, but only methods.  So there are no
 right places where nested methods are defined.  However, if
 refinements are introduced, a refinement enabled only in the outer
 method would be the right place.  For example, the above code is
 almost equivalent to the following code:

   def foo
     klass = self.class
     m = Module.new {
       refine klass do
         def bar
           puts "bar"
         end
       end
     }
     using m
     bar
   end
   foo  #=> bar
   bar  #=> NoMethodError

 The attached patch is based on SVN trunk r29837.
=end

On 20.11.2012 03:45, shugo (Shugo Maeda) wrote:
> I think optional features of Refinements are as follows:
>
> A. refinement inheritance in class hierarchies

I generally think that class/module inheritance is the wrong propagation
strategy for refinements. If you see refinements as monkey patches they
are only necessary to get *your own code* working as desired. When you
provide an abstract class/module in a library that application code can
inherit from, then the application itself may not need the refinements
you use to make the internals of your class tick.

In other words, module/class inheritance is about inheriting desired
*behavior*. Monkey patches may not be desired behavior, they are the
dirty internal mechanics that should be hidden from subclasses.

There is an axis orthogonal to behavior. It's responsibility. This
second axis is generally associated with the module namespaces. E.g. I
expect the rails maintainers to be responsible for the ::ActiveRecord
namespace and be aware of what their own code is doing. Their own
refinements should not pose a problem to them. But they might be for me.

If they "need" a String.camelize in their code then they should be able
to add it without polluting my code. If I consider it useful I can still
include their refinements into my code.

Therefore I think that class inheritance should be removed. And if it
gets replaced in the future then it should be with submodule based
inheritance.

The other issue i have with inheritance is that there is no opt-out.

This is the very same issue we're trying to fix! If some piece of code
monkey-patches Object then the whole application is hit by that
modification. Refinements are supposed to prevent this. But what happens
if i want to use a module that applies refinements? Then I would get hit
by those refinements too.

If we want inheritance then we need some way to opt-out of refinements.
Consider someone applying "using" to Object itself early on during the
application loading process. We would be in the same mess we are in now.
Actually. It would be worse, some methods might see the refinements and
others don't, depending on their definition time.


For now people can use Module.extended/.included if they really want to
add refinement inheritance themselves.

> B. refinement activation for reopened module definitions
> C. refinement activation for the string version of module_eval/instance_eval
> D. refinement activation for the block version of module_eval/instance_eval

I don't feel strongly about those, but if the module_eval performance
really has such a big issue as headius asserts then it might be better
to postpone it until a solution has been found.

Probably the safest approach for now would be to use the source
refinement scope (which is quasi-static) for module_eval by default and
add a way to use the target scope (or an explicit scope) later on as
needed. If there is any performance impact it would restricted to the
target-scoped procs.

I think some clever optimizations should be able to eliminate cases
where procs flow through consistent code paths, i.e. are always
evaluated against the same target refinement scope as usually is the
case with DSLs or class-level configurations.



>    end
>    end
>
> I think it's the most important feature of Refinements.  Without it, it's hard 
to avoid conflicts among multiple refinements.

What about cases like

   module SomeExt
     refine String do
       def bar
       end
     end
   end

   class Foo
     using SomeExt

     def self.test1
       "".tap(&:bar)
     end

     def self.test2
       "".tap{|f| f.bar}
     end
   end


String.bar is only visible inside Foo, but in test1 the Proc is created
in .to_proc of Symbol, i.e. on a different stack frame, which shouldn't
be able to see bar due to the scoping. Which leads to counter-intuitive
results.