Forum: Ruby-core Refinements and nested methods

Issue #4085 has been updated by shugo (Shugo Maeda).


Thanks for your answering my questions.  I understand your intention.

matz wrote:
> In message "Re: [ruby-core:50299] [ruby-trunk - Feature #4085] Refinements and 
nested methods"
>      on Thu, 29 Nov 2012 15:02:03 +0900, "shugo (Shugo Maeda)" 
<redmine@ruby-lang.org> writes:
>
>  |> * refinements are file scope
>  |> * only top-level "using" is available
>  |> * no module scope refinement
>  |
>  |Do these constraints just mean that main.using is available, but Module#using 
is not?
>
>  Yes, only main.using should be available. No Module#using (for 2.0).

So, the current behavior of main.using need not be changed, right?
Technically, the current behavior of main.using is not file scope.
The scope of main.using is from the point where using is *called at 
runtime* to the end of that file.
For example,

p 1 / 2     # => 0 (not refined)
using MathN # MathN refines Fixnum#/
p 1 / 2     # => (1/2) (refined)

And a more complex example is:

if false
  using MathN
end
p 1 / 2     #=> 0 (not refined)

>  |Currently, (a) prints :foo, and (b) raises a NoMethodError.
>
>  Refinements will be available only from:
>
>  * the scope where refinements are added by calling "using"
>  * or inside of refine blocks
>
>  Inside of refine blocks (not whole module scope) might be
>  controversial, but I think it's OK to restrict refinements there.  As
>  a result, both (a) and (b) raise NoMethodError.  But "".foo can be called
>  from within the refine block.

"Inside of refine blocks" means that all refinements defined in a module 
are activated
in all refine blocks in that module and nested modules, right?

There are some considerations:

* It may be better not to support nested module to simplify things.
  I think support for nested modules are important for refinement users, 
but are not so important
  for refinement authors.
* It may be hard to implement efficiently activation of refinements 
which are defined after refine blocks.
  For example,
    module M
      refine Array do
        def to_json; "[" + map { |i| i.to_json } + "]" end
      end

      refine Hash do
        def to_json; "{" + map { |k, v| k.to_s.dump + ":" + v.to_json } 
+ "}" end
      end
    end
  It may be better to limit refinement activation in refine blocks to 
refinements to be defined by the refine blocks theselves, to simplify 
things.  If there's no refinement activation in refine blocks, recursive 
methods cannot be defined, so the refinement to be defined should be 
activated at least.
* Refinement activation in refine blocks may have the same problem as 
refinement-aware module_eval,
  for example in the following code:

    F = Proc.new { 1 / 2 }
    module M
      refine Fixnum do def /(other) quo(other) end end
      refine(Object, &F)
    end

>  |There are two aspects about refinement addition.  They are defined in modules 
by Module#refine, and activated in certain scopes by using.
>  |Does "to add refinements" mean to define (or inherit indirectly) refinements 
in modules, or to activate refinements in modules, or both of them?
>
>  I meant included module will provide refinement of combination of including
>  module(s) and the module itself.

It does make sense.
What happens if the same class is refined both in a module and another 
module included into that module?
For example,

  class C
    def foo; p :C; end
  end
  module M1
    refine String do def foo; p :M1; super; end; end
  end
  module M2
    include M1
    refine String do def foo; p :M2; super; end; end
  end
  using M2
  C.new.foo #=> ?

I think it's better to just calls M2 and C, not M1, to simplify things.
super chain is too complex here.

>  |Finally, how super in refinements should behave in the new spec?
>
>  Refinements should come before normal methods, so super in the normal
>  method will not see a refined method, and super in the refined method
>  will see a normal method (or other refined method if refinements are
>  stacked).

Charles showed an edge case.

module A; refine(Numeric) { def blah; puts 'A'; super; end }; end
module B; refine(Integer) { def blah; puts 'B'; super; end }; end
module C; refine(Fixnum) { def blah; puts 'C'; super; end }; end
using A; using B; using C
1.blah

Currently, only C is called, but what should happen?
At first, I thought all blah should be called, but super in C is in 
scope of neither A nor B,
it might be better not to call A and B.

I'm starting to think it might be better to limit super to call only the 
original method in the refined class to simplify the spec.

For example,

class X; def blah; puts 'X'; end
module A; refine(X) { def blah; puts 'A'; super; end }; end
module B; refine(X) { def blah; puts 'B'; super; end }; end
module C; refine(X) { def blah; puts 'C'; super; end }; end
using A; using B; using C
1.blah

Only C and X is called in the above code.
At first, I thought that stacking refinements and super chain are useful 
for aspect oriented programming.
But refinements have no local rebinding, so it might not be a real use 
case of refinements.

----------------------------------------
Feature #4085: Refinements and nested methods
https://bugs.ruby-lang.org/issues/4085#change-34168

Author: shugo (Shugo Maeda)
Status: Assigned
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category: core
Target version: 2.0.0


=begin
 As I said at RubyConf 2010, I'd like to propose a new features called
 "Refinements."

 Refinements are similar to Classboxes.  However, Refinements doesn't
 support local rebinding as mentioned later.  In this sense,
 Refinements might be more similar to selector namespaces, but I'm not
 sure because I have never seen any implementation of selector
 namespaces.

 In Refinements, a Ruby module is used as a namespace (or classbox) for
 class extensions.  Such class extensions are called refinements.  For
 example, the following module refines Fixnum.

   module MathN
     refine Fixnum do
       def /(other) quo(other) end
     end
   end

 Module#refine(klass) takes one argument, which is a class to be
 extended.  Module#refine also takes a block, where additional or
 overriding methods of klass can be defined.  In this example, MathN
 refines Fixnum so that 1 / 2 returns a rational number (1/2) instead
 of an integer 0.

 This refinement can be enabled by the method using.

   class Foo
     using MathN

     def foo
       p 1 / 2
     end
   end

   f = Foo.new
   f.foo #=> (1/2)
   p 1 / 2

 In this example, the refinement in MathN is enabled in the definition
 of Foo.  The effective scope of the refinement is the innermost class,
 module, or method where using is called; however the refinement is not
 enabled before the call of using.  If there is no such class, module,
 or method, then the effective scope is the file where using is called.
 Note that refinements are pseudo-lexically scoped.  For example,
 foo.baz prints not "FooExt#bar" but "Foo#bar" in the following code:

   class Foo
     def bar
       puts "Foo#bar"
     end

     def baz
       bar
     end
   end

   module FooExt
     refine Foo do
       def bar
         puts "FooExt#bar"
       end
     end
   end

   module Quux
     using FooExt

     foo = Foo.new
     foo.bar  # => FooExt#bar
     foo.baz  # => Foo#bar
   end

 Refinements are also enabled in reopened definitions of classes using
 refinements and definitions of their subclasses, so they are
 *pseudo*-lexically scoped.

   class Foo
     using MathN
   end

   class Foo
     # MathN is enabled in a reopened definition.
     p 1 / 2  #=> (1/2)
   end

   class Bar < Foo
     # MathN is enabled in a subclass definition.
     p 1 / 2  #=> (1/2)
   end

 If a module or class is using refinements, they are enabled in
 module_eval, class_eval, and instance_eval if the receiver is the
 class or module, or an instance of the class.

   module A
     using MathN
   end
   class B
     using MathN
   end
   MathN.module_eval do
     p 1 / 2  #=> (1/2)
   end
   A.module_eval do
     p 1 / 2  #=> (1/2)
   end
   B.class_eval do
     p 1 / 2  #=> (1/2)
   end
   B.new.instance_eval do
     p 1 / 2  #=> (1/2)
   end

 Besides refinements, I'd like to propose new behavior of nested 
methods.
 Currently, the scope of a nested method is not closed in the outer 
method.

   def foo
     def bar
       puts "bar"
     end
     bar
   end
   foo  #=> bar
   bar  #=> bar

 In Ruby, there are no functions, but only methods.  So there are no
 right places where nested methods are defined.  However, if
 refinements are introduced, a refinement enabled only in the outer
 method would be the right place.  For example, the above code is
 almost equivalent to the following code:

   def foo
     klass = self.class
     m = Module.new {
       refine klass do
         def bar
           puts "bar"
         end
       end
     }
     using m
     bar
   end
   foo  #=> bar
   bar  #=> NoMethodError

 The attached patch is based on SVN trunk r29837.
=end

In message "Re: [ruby-core:50348] [ruby-trunk - Feature #4085] 
Refinements and nested methods"
    on Fri, 30 Nov 2012 10:00:34 +0900, "shugo (Shugo Maeda)" 
<redmine@ruby-lang.org> writes:

|>  Yes, only main.using should be available. No Module#using (for 2.0).
|
|So, the current behavior of main.using need not be changed, right?
|Technically, the current behavior of main.using is not file scope.
|The scope of main.using is from the point where using is *called at runtime* to 
the end of that file.
|For example,
|
|p 1 / 2     # => 0 (not refined)
|using MathN # MathN refines Fixnum#/
|p 1 / 2     # => (1/2) (refined)
|
|And a more complex example is:
|
|if false
|  using MathN
|end
|p 1 / 2     #=> 0 (not refined)

In that sense, I don't think we need to change the behavior.  What I
meant by "file scope" was that refinement only available until end of
the file.

|"Inside of refine blocks" means that all refinements defined in a module are 
activated
|in all refine blocks in that module and nested modules, right?

Yes, that what I meant.

|There are some considerations:
|
|* It may be better not to support nested module to simplify things.
|  I think support for nested modules are important for refinement users, but are 
not so important
|  for refinement authors.

I don't think nested modules are important.  We can drop them, at
least for 2.0 to simplify things.

|* It may be hard to implement efficiently activation of refinements which are 
defined after refine blocks.
|  For example,
|    module M
|      refine Array do
|        def to_json; "[" + map { |i| i.to_json } + "]" end
|      end
|
|      refine Hash do
|        def to_json; "{" + map { |k, v| k.to_s.dump + ":" + v.to_json } + "}" end
|      end
|    end
|  It may be better to limit refinement activation in refine blocks to refinements 
to be defined by the refine blocks theselves, to simplify things.  If there's no 
refinement activation in refine blocks, recursive methods cannot be defined, so 
the refinement to be defined should be activated at least.

I am not sure if I understand you correctly.  I thought method look-up
should be done in run-time.  So only I can say it that refinement M
will be available in both refine blocks in the above example.

|* Refinement activation in refine blocks may have the same problem as 
refinement-aware module_eval,
|  for example in the following code:
|
|    F = Proc.new { 1 / 2 }
|    module M
|      refine Fixnum do def /(other) quo(other) end end
|      refine(Object, &F)
|    end

Yes, but I consider its behavior implementation dependent.
Fundamentally you cannot expect passing proc to lambda work correctly.

|What happens if the same class is refined both in a module and another module 
included into that module?
|For example,
|
|  class C
|    def foo; p :C; end
|  end
|  module M1
|    refine String do def foo; p :M1; super; end; end
|  end
|  module M2
|    include M1
|    refine String do def foo; p :M2; super; end; end
|  end
|  using M2
|  C.new.foo #=> ?
|
|I think it's better to just calls M2 and C, not M1, to simplify things.
|super chain is too complex here.

I was thinking of M2->M1->C, but M2->C is simpler and acceptable.

|Charles shown an edge case.
|
|module A; refine(Numeric) { def blah; puts 'A'; super; end }; end
|module B; refine(Integer) { def blah; puts 'B'; super; end }; end
|module C; refine(Fixnum) { def blah; puts 'C'; super; end }; end
|using A; using B; using C
|1.blah
|
|Currently, only C is called, but what should happen?
|At first, I thought all blah should be called, but super in C is in scope of 
neither A nor B,
|it might be better not to call A and B.
|
|I'm starting to think it might be better to limit super to call only the original 
method in the refined class to simplify the spec.

So do you mean refined method appear only once in method look-up chain?

|For example,
|
|class X; def blah; puts 'X'; end
|module A; refine(X) { def blah; puts 'A'; super; end }; end
|module B; refine(X) { def blah; puts 'B'; super; end }; end
|module C; refine(X) { def blah; puts 'C'; super; end }; end
|using A; using B; using C
|1.blah

X.new.blah # <= do you mean X here?

|Only C and X is called in the above code.
|At first, I thought that stacking refinements and super chain are useful for 
aspect oriented programming.
|But refinements have no local rebinding, so it might not be a real use case of 
refinements.

Fair enough. If we can warn users for conflicted refinements like
above, it's even better.

              matz.

Em 29-11-2012 23:42, Yukihiro Matsumoto escreveu:
> |  using M2
> |  C.new.foo #=>  ?
> |
> |I think it's better to just calls M2 and C, not M1, to simplify things.
> |super chain is too complex here.
>
> I was thinking of M2->M1->C, but M2->C is simpler and acceptable.

I'm worried about this. We shouldn't be defining this behavior in light
of what is simple/feasible until 2.0 release. Because if this decision
changes later it will be backward incompatible and it is most likely
that we won't be able to change this behavior in the future to keep
compatibility.

It would be better to think well about this now. What is the really
desired behavior in the long-run to avoid compatibility issues in the
future?