Forum: Ruby-core Refinements and nested methods

F1d37642fdaa1662ff46e4c65731e9ab?d=identicon&s=25 Charles Nutter (headius)
on 2012-11-29 20:05
(Received via mailing list)
Issue #4085 has been updated by headius (Charles Nutter).


Anonymous wrote:
>  |
>  from within the refine block.
I assume this is so that refinement methods will see each other, yes?

module X
  refine String do
    def foo; :foo; end
    def bar; foo; end # should be able to call foo
  end
end

So all call sites within a refine block must look for refinements.
Now...I would assume they can only see refinements in the same block,
but...

>  >  module M
>  >    refine Array do
>  >      "".foo  # => OK
>  >    end
>  >  end
>  >end
>  >
>  >using R::M
>  > "".foo  # => NG

This is incredibly confusing to me. Why are the String refinements
active within the refine Array block? That module:

* Is not within the refine String block
* Is not within a file that uses the String refinement
* Does not refine String

Why can it see the String refinements even though it's not in a "using"
file and not within a refine String block?

>  I meant included module will provide refinement of combination of including
>  module(s) and the module itself.

This seems mostly reasonable, since modules that include modules carry
all those modules with them to the eventual include site. So using
basically considers the entire hierarchy of modules.

>  |  module R2
>  Since all calls of "".foo and "".bar are outside of refine blocks,
>  they should raise NoMethodError.  But R2 should provide refinement to
>  add method #bar and #foo to String class.

More interesting, given the above modules... I assume this would work:

#some file that's loaded the above refinements
using R2

"".foo # => OK
"".bar # => OK

>  |Finally, how super in refinements should behave in the new spec?
>
>  Refinements should come before normal methods, so super in the normal
>  method will not see a refined method, and super in the refined method
>  will see a normal method (or other refined method if refinements are
>  stacked).

By stacking, I assume you mean something like this:

class String
  def bar; p :base; end
end

module M1
  refine String do
    def bar; p :m1; super end
  end
end

module M2
  refine String do
    def bar; p :m2; super; end
  end
end

using M1
using M2

"".bar # => should print :m2, :m1, :base

Correct?

My idea of *actually* putting refinements into the hierarchy seems like
it might fit this best, rather than searching two hierarchies. Will have
to think about it more.

>  Any more questions?

Yes, lots! :)

* Will we make more methods have special access to the caller's frame to
support refinements? I'm speaking here of modifications to send, method
reflection, to_proc, and so on. Of these, send sorta has access to
caller frame, but it's unspecified (send :binding behaves differently on
different impls). The reflection methods (method, instance_method, ...)
and to_proc do not currently have access to caller's frame, and would
have to be special-cased (and in JRuby that would force deoptimization
of code that calls them to ensure we have a frame available).  I would
argue that *none* of these methods should see the effects of
refinements, because they are not defined within a refined file or
scope. Making them see refinements extends refined effects down-stack in
unpredictable ways.

* Questions about about the scoping of refinements in seemingly
unrelated refine blocks.

* Performance expectations. Is performance being completely ignored
here, or shall we set some goals for the implementation? I appreciate
your desire to make Ruby more expressive, but that doesn't pay the bills
at the end of the day if refined calls are 100x slower than regular
calls. Some consideration of performance needs to be made up front. I
would propose that unless refined calls can be made exactly the same
speed as unrefined calls, they should not be included. We'd essentially
be adding a file-scoped feature that hurts performance of all calls in
that file. Expressivity doesn't trump slowing an entire system down
because of a single call made at the top level of a file.

I'm glad to see refinements being reduced in scope, but we still have a
lot to talk through. I share others' concerns about this feature not
being settled just a couple months before 2.0 is supposed to be
released, and at this point I'd still rather see it delayed to 2.1 so we
can work through all the edges. I do not yet have a clear picture in my
head of how refinements are supposed to be structured, and will have to
reboot the understanding I did have based on recent updates.

I will try to modify my refinements implementation to work according to
the new specification and see how it feels, implementation-wise.

FWIW, I made an improvement to JRuby 1.7.2 that makes the calling scope
(the static part where cref and refinements live) available to all
methods, even when scope/frame have been optimized away. It may make it
possible to implement refinements without reducing the number of
optimizations we do.

To others on this thread: I'd really like to hear more substantive input
on the feature, rather than just "please don't do this, it's bad, I
don't like it, it's too late, wah wah." If you have a specific aspect
you don't like, discuss that. If you don't like the feature in toto,
explain why. Special pleading for it to be removed does not further the
conversation.
----------------------------------------
Feature #4085: Refinements and nested methods
https://bugs.ruby-lang.org/issues/4085#change-34162

Author: shugo (Shugo Maeda)
Status: Assigned
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category: core
Target version: 2.0.0


=begin
 As I said at RubyConf 2010, I'd like to propose a new features called
 "Refinements."

 Refinements are similar to Classboxes.  However, Refinements doesn't
 support local rebinding as mentioned later.  In this sense,
 Refinements might be more similar to selector namespaces, but I'm not
 sure because I have never seen any implementation of selector
 namespaces.

 In Refinements, a Ruby module is used as a namespace (or classbox) for
 class extensions.  Such class extensions are called refinements.  For
 example, the following module refines Fixnum.

   module MathN
     refine Fixnum do
       def /(other) quo(other) end
     end
   end

 Module#refine(klass) takes one argument, which is a class to be
 extended.  Module#refine also takes a block, where additional or
 overriding methods of klass can be defined.  In this example, MathN
 refines Fixnum so that 1 / 2 returns a rational number (1/2) instead
 of an integer 0.

 This refinement can be enabled by the method using.

   class Foo
     using MathN

     def foo
       p 1 / 2
     end
   end

   f = Foo.new
   f.foo #=> (1/2)
   p 1 / 2

 In this example, the refinement in MathN is enabled in the definition
 of Foo.  The effective scope of the refinement is the innermost class,
 module, or method where using is called; however the refinement is not
 enabled before the call of using.  If there is no such class, module,
 or method, then the effective scope is the file where using is called.
 Note that refinements are pseudo-lexically scoped.  For example,
 foo.baz prints not "FooExt#bar" but "Foo#bar" in the following code:

   class Foo
     def bar
       puts "Foo#bar"
     end

     def baz
       bar
     end
   end

   module FooExt
     refine Foo do
       def bar
         puts "FooExt#bar"
       end
     end
   end

   module Quux
     using FooExt

     foo = Foo.new
     foo.bar  # => FooExt#bar
     foo.baz  # => Foo#bar
   end

 Refinements are also enabled in reopened definitions of classes using
 refinements and definitions of their subclasses, so they are
 *pseudo*-lexically scoped.

   class Foo
     using MathN
   end

   class Foo
     # MathN is enabled in a reopened definition.
     p 1 / 2  #=> (1/2)
   end

   class Bar < Foo
     # MathN is enabled in a subclass definition.
     p 1 / 2  #=> (1/2)
   end

 If a module or class is using refinements, they are enabled in
 module_eval, class_eval, and instance_eval if the receiver is the
 class or module, or an instance of the class.

   module A
     using MathN
   end
   class B
     using MathN
   end
   MathN.module_eval do
     p 1 / 2  #=> (1/2)
   end
   A.module_eval do
     p 1 / 2  #=> (1/2)
   end
   B.class_eval do
     p 1 / 2  #=> (1/2)
   end
   B.new.instance_eval do
     p 1 / 2  #=> (1/2)
   end

 Besides refinements, I'd like to propose new behavior of nested
methods.
 Currently, the scope of a nested method is not closed in the outer
method.

   def foo
     def bar
       puts "bar"
     end
     bar
   end
   foo  #=> bar
   bar  #=> bar

 In Ruby, there are no functions, but only methods.  So there are no
 right places where nested methods are defined.  However, if
 refinements are introduced, a refinement enabled only in the outer
 method would be the right place.  For example, the above code is
 almost equivalent to the following code:

   def foo
     klass = self.class
     m = Module.new {
       refine klass do
         def bar
           puts "bar"
         end
       end
     }
     using m
     bar
   end
   foo  #=> bar
   bar  #=> NoMethodError

 The attached patch is based on SVN trunk r29837.
=end
6738588a11b852833edf6aec90ef6fa3?d=identicon&s=25 Yukihiro Matsumoto (Guest)
on 2012-11-29 22:14
(Received via mailing list)
In message "Re: [ruby-core:50338] [ruby-trunk - Feature #4085]
Refinements and nested methods"
    on Fri, 30 Nov 2012 04:05:15 +0900, "headius (Charles Nutter)"
<headius@headius.com> writes:

|I assume this is so that refinement methods will see each other, yes?

Yes.

|module X
|  refine String do
|    def foo; :foo; end
|    def bar; foo; end # should be able to call foo
|  end
|end
|
|So all call sites within a refine block must look for refinements. Now...I would
assume they can only see refinements in the same block, but...

No, whole refinement provided by module X can be seen from refine
block in module X.

|This is incredibly confusing to me. Why are the String refinements active within
the refine Array block? That module:
|
|* Is not within the refine String block
|* Is not within a file that uses the String refinement
|* Does not refine String
|
|Why can it see the String refinements even though it's not in a "using" file and
not within a refine String block?

Because it's within the body of refinement module X. Since both
refinements are defined in the same refinement, we consider they are
related (you can't see the relation from this silly examples).

|>  Refinements should come before normal methods, so super in the normal
|>  method will not see a refined method, and super in the refined method
|>  will see a normal method (or other refined method if refinements are
|>  stacked).
|
|By stacking, I assume you mean something like this:
|
|class String
|  def bar; p :base; end
|end
|
|module M1
|  refine String do
|    def bar; p :m1; super end
|  end
|end
|
|module M2
|  refine String do
|    def bar; p :m2; super; end
|  end
|end
|
|using M1
|using M2
|
|"".bar # => should print :m2, :m1, :base
|
|Correct?

Correct.

|My idea of *actually* putting refinements into the hierarchy seems like it might
fit this best, rather than searching two hierarchies. Will have to think about 
it
more.

It makes refinement decoration quite unpredictable.  For example,

  module M
    refine Integer do
      def /(n); self / n.to_f; end
    end
  end
  using M
  1 / 2 # => you expect 0.5

But if refinements are searched through inheritaance hierarchy, it
won't work since there's Fixnum#/.

|>  Any more questions?
|
|Yes, lots! :)

I will answer each of them later.

              matz.
6738588a11b852833edf6aec90ef6fa3?d=identicon&s=25 Yukihiro Matsumoto (Guest)
on 2012-11-30 02:23
(Received via mailing list)
In message "Re: [ruby-core:50338] [ruby-trunk - Feature #4085]
Refinements and nested methods"
    on Fri, 30 Nov 2012 04:05:15 +0900, "headius (Charles Nutter)"
<headius@headius.com> writes:

|>  Any more questions?
|
|Yes, lots! :)
|
|* Will we make more methods have special access to the caller's frame to support
refinements? I'm speaking here of modifications to send, method reflection,
to_proc, and so on. Of these, send sorta has access to caller frame, but it's
unspecified (send :binding behaves differently on different impls). The 
reflection
methods (method, instance_method, ...) and to_proc do not currently have access 
to
caller's frame, and would have to be special-cased (and in JRuby that would 
force
deoptimization of code that calls them to ensure we have a frame available).  I
would argue that *none* of these methods should see the effects of refinements,
because they are not defined within a refined file or scope. Making them see
refinements extends refined effects down-stack in unpredictable ways.

From usability perspective, all retrospective methods e.g. respond_to?
methods etc. should reflect refinements. But for 2.0, I don't make
them check refinements, because of performance and complexity.  It
should be issue of future version.

|* Questions about about the scoping of refinements in seemingly unrelated refine
blocks.
|
|* Performance expectations. Is performance being completely ignored here, or
shall we set some goals for the implementation? I appreciate your desire to make
Ruby more expressive, but that doesn't pay the bills at the end of the day if
refined calls are 100x slower than regular calls. Some consideration of
performance needs to be made up front. I would propose that unless refined calls
can be made exactly the same speed as unrefined calls, they should not be
included. We'd essentially be adding a file-scoped feature that hurts 
performance
of all calls in that file. Expressivity doesn't trump slowing an entire system
down because of a single call made at the top level of a file.

My expectation is that if you don't see "using" and "refine" in the
source code, the performance will not be affected, as much as
possible.  It's the reason I gave up refinements in the module bodies,
inherited modules/classes and module_eval.  I don't demamd refined
call to be exact same speed as unrefined ones, but I'd expect them not
significantly slower.  I'd accept them if they are 5-15% slower than
normal calls.

              matz.
This topic is locked and can not be replied to.