Forum: Ruby-core [ruby-trunk - Feature #9112][Open] Make module lookup more dynamic (Including modules into a module

8db112f19ce0a69c7e490bbf27256346?d=identicon&s=25 PragTob (Tobias Pfeiffer) (Guest)
on 2013-11-14 20:26
(Received via mailing list)
Issue #9112 has been reported by PragTob (Tobias Pfeiffer).

----------------------------------------
Feature #9112: Make module lookup more dynamic (Including modules into a
module after it has already been included)
https://bugs.ruby-lang.org/issues/9112

Author: PragTob (Tobias Pfeiffer)
Status: Open
Priority: Normal
Assignee:
Category:
Target version:


If a module (M) is included into a class (C) and afterwards another
module (M2) is included into the first module (M) then C does not
include M2 and instances do not respond to methods defined in M2. I
think instances of C should respond to methods defined in M2 and C
should include M2.

I created a gist detailing the problem I have:
https://gist.github.com/PragTob/7472643

I think this behavior is confusing, because if I'd reopen module M and
just add methods there then instances of C can call those methods.
However if I include another module in M then instances of C can not
call those methods.

Any opinions on if this would be a better behavior or why it isn't?

(was unsure to file it as a bug or feature)
58479f76374a3ba3c69b9804163f39f4?d=identicon&s=25 drbrain (Eric Hodel) (Guest)
on 2013-11-15 01:24
(Received via mailing list)
Issue #9112 has been updated by drbrain (Eric Hodel).

Category set to core
Status changed from Open to Assigned
Assignee set to matz (Yukihiro Matsumoto)

This would be a new feature.

I believe this was requested and rejected back in the 1.8 days, but
looking for the discussion is too difficult for me.

If it was, maybe matz has changed his mind
----------------------------------------
Feature #9112: Make module lookup more dynamic (Including modules into a
module after it has already been included)
https://bugs.ruby-lang.org/issues/9112#change-42943

Author: PragTob (Tobias Pfeiffer)
Status: Assigned
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category: core
Target version:


If a module (M) is included into a class (C) and afterwards another
module (M2) is included into the first module (M) then C does not
include M2 and instances do not respond to methods defined in M2. I
think instances of C should respond to methods defined in M2 and C
should include M2.

I created a gist detailing the problem I have:
https://gist.github.com/PragTob/7472643

I think this behavior is confusing, because if I'd reopen module M and
just add methods there then instances of C can call those methods.
However if I include another module in M then instances of C can not
call those methods.

Any opinions on if this would be a better behavior or why it isn't?

(was unsure to file it as a bug or feature)
F1d6cc2b735bfd82c8773172da2aeab9?d=identicon&s=25 Nobuyoshi Nakada (nobu)
on 2013-11-15 02:14
(Received via mailing list)
Issue #9112 has been updated by nobu (Nobuyoshi Nakada).


It has been rejected because it is hard to implement without significant
performance penalty.
Now we have subclass trees, so it might be possible.
----------------------------------------
Feature #9112: Make module lookup more dynamic (Including modules into a
module after it has already been included)
https://bugs.ruby-lang.org/issues/9112#change-42946

Author: PragTob (Tobias Pfeiffer)
Status: Assigned
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category: core
Target version:


If a module (M) is included into a class (C) and afterwards another
module (M2) is included into the first module (M) then C does not
include M2 and instances do not respond to methods defined in M2. I
think instances of C should respond to methods defined in M2 and C
should include M2.

I created a gist detailing the problem I have:
https://gist.github.com/PragTob/7472643

I think this behavior is confusing, because if I'd reopen module M and
just add methods there then instances of C can call those methods.
However if I include another module in M then instances of C can not
call those methods.

Any opinions on if this would be a better behavior or why it isn't?

(was unsure to file it as a bug or feature)
8db112f19ce0a69c7e490bbf27256346?d=identicon&s=25 PragTob (Tobias Pfeiffer) (Guest)
on 2013-11-17 14:40
(Received via mailing list)
Issue #9112 has been updated by PragTob (Tobias Pfeiffer).


Thank you for the quick reply Eric and Nobu! Looking forward to how this
turns out :-)


----------------------------------------
Feature #9112: Make module lookup more dynamic (Including modules into a
module after it has already been included)
https://bugs.ruby-lang.org/issues/9112#change-42987

Author: PragTob (Tobias Pfeiffer)
Status: Assigned
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category: core
Target version:


If a module (M) is included into a class (C) and afterwards another
module (M2) is included into the first module (M) then C does not
include M2 and instances do not respond to methods defined in M2. I
think instances of C should respond to methods defined in M2 and C
should include M2.

I created a gist detailing the problem I have:
https://gist.github.com/PragTob/7472643

I think this behavior is confusing, because if I'd reopen module M and
just add methods there then instances of C can call those methods.
However if I include another module in M then instances of C can not
call those methods.

Any opinions on if this would be a better behavior or why it isn't?

(was unsure to file it as a bug or feature)
D9ebdcb66f1583378e6f72155db507e2?d=identicon&s=25 Hans Mackowiak (hanmac)
on 2013-11-17 15:39
(Received via mailing list)
Issue #9112 has been updated by Hanmac (Hans Mackowiak).


would it with the change also possible to add Module multible times into
the Ancestor chain?
like
  C1 < M1 < C2 < M1 < M2 < C3
?
----------------------------------------
Feature #9112: Make module lookup more dynamic (Including modules into a
module after it has already been included)
https://bugs.ruby-lang.org/issues/9112#change-42988

Author: PragTob (Tobias Pfeiffer)
Status: Assigned
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category: core
Target version:


If a module (M) is included into a class (C) and afterwards another
module (M2) is included into the first module (M) then C does not
include M2 and instances do not respond to methods defined in M2. I
think instances of C should respond to methods defined in M2 and C
should include M2.

I created a gist detailing the problem I have:
https://gist.github.com/PragTob/7472643

I think this behavior is confusing, because if I'd reopen module M and
just add methods there then instances of C can call those methods.
However if I include another module in M then instances of C can not
call those methods.

Any opinions on if this would be a better behavior or why it isn't?

(was unsure to file it as a bug or feature)
F1d6cc2b735bfd82c8773172da2aeab9?d=identicon&s=25 Nobuyoshi Nakada (nobu)
on 2013-11-18 03:39
(Received via mailing list)
Issue #9112 has been updated by nobu (Nobuyoshi Nakada).


Hanmac (Hans Mackowiak) wrote:
> would it with the change also possible to add Module multible times into the
Ancestor chain?

I'd tried it once, but found that some modules implicitly expect they
are included only one time.
So, we'll need a way to tell a module can be included multiple times or
not, perhaps.
----------------------------------------
Feature #9112: Make module lookup more dynamic (Including modules into a
module after it has already been included)
https://bugs.ruby-lang.org/issues/9112#change-42991

Author: PragTob (Tobias Pfeiffer)
Status: Assigned
Priority: Normal
Assignee: matz (Yukihiro Matsumoto)
Category: core
Target version:


If a module (M) is included into a class (C) and afterwards another
module (M2) is included into the first module (M) then C does not
include M2 and instances do not respond to methods defined in M2. I
think instances of C should respond to methods defined in M2 and C
should include M2.

I created a gist detailing the problem I have:
https://gist.github.com/PragTob/7472643

I think this behavior is confusing, because if I'd reopen module M and
just add methods there then instances of C can call those methods.
However if I include another module in M then instances of C can not
call those methods.

Any opinions on if this would be a better behavior or why it isn't?

(was unsure to file it as a bug or feature)
F1d37642fdaa1662ff46e4c65731e9ab?d=identicon&s=25 Charles Nutter (headius)
on 2014-10-06 22:33
(Received via mailing list)
Issue #9112 has been updated by Charles Nutter.


nobu's point about reincluding the same module brought up a different
concern.

Many modules perform very specific actions at include time. If they did
not go through a reinclude process for every hierarchy, those changes
might not happen. For example:

module X
  def self.included(target)
    ... make changes to target hierarchy in addition to the include
  end
end

module Y
  include X # incuded runs against Y hierarchy
end

class A
  include Y # included for both Y and X run under current behavior, but
only Y would run under new behavior
end

----------------------------------------
Feature #9112: Make module lookup more dynamic (Including modules into a
module after it has already been included)
https://bugs.ruby-lang.org/issues/9112#change-49232

* Author: Tobias Pfeiffer
* Status: Assigned
* Priority: Normal
* Assignee: Yukihiro Matsumoto
* Category: core
* Target version:
----------------------------------------
If a module (M) is included into a class (C) and afterwards another
module (M2) is included into the first module (M) then C does not
include M2 and instances do not respond to methods defined in M2. I
think instances of C should respond to methods defined in M2 and C
should include M2.

I created a gist detailing the problem I have:
https://gist.github.com/PragTob/7472643

I think this behavior is confusing, because if I'd reopen module M and
just add methods there then instances of C can call those methods.
However if I include another module in M then instances of C can not
call those methods.

Any opinions on if this would be a better behavior or why it isn't?

(was unsure to file it as a bug or feature)
7223c62b7310e164eb79c740188abbda?d=identicon&s=25 Xavier Noria (fxn)
on 2014-10-06 22:54
(Received via mailing list)
Issue #9112 has been updated by Xavier Noria.


+1 over here.

I remember I discovered this the hard-way extracting some code from
Rails into a gem.

I believe the current behavior is a leak of the implementation.
Conceptually, in my opinion, a method or constant lookup should follow
the ancestor chain at that point in runtime. That way you can add
behavior to modules adding method directly or via another module for
organizational purposes.

I have always seen the linearization of ancestors as an implementation
detail not belonging to the conceptual object model of Ruby. From the
user perspective it does not matter if there's a pointer to super, or if
there's a tree search of some sort going on.

That's my view, don't really know if it's also the view of Ruby core.

I asked Matz about this in the past and he said what was said above,
that if there was no performance penalty it would be open to this
change. See

https://www.ruby-forum.com/topic/1458576


----------------------------------------
Feature #9112: Make module lookup more dynamic (Including modules into a
module after it has already been included)
https://bugs.ruby-lang.org/issues/9112#change-49233

* Author: Tobias Pfeiffer
* Status: Assigned
* Priority: Normal
* Assignee: Yukihiro Matsumoto
* Category: core
* Target version:
----------------------------------------
If a module (M) is included into a class (C) and afterwards another
module (M2) is included into the first module (M) then C does not
include M2 and instances do not respond to methods defined in M2. I
think instances of C should respond to methods defined in M2 and C
should include M2.

I created a gist detailing the problem I have:
https://gist.github.com/PragTob/7472643

I think this behavior is confusing, because if I'd reopen module M and
just add methods there then instances of C can call those methods.
However if I include another module in M then instances of C can not
call those methods.

Any opinions on if this would be a better behavior or why it isn't?

(was unsure to file it as a bug or feature)
87bf8ebf42753b77da12f70e9d1e5671?d=identicon&s=25 unknown (Guest)
on 2014-10-07 17:29
(Received via mailing list)
Issue #9112 has been updated by Timur Duehr.


I'm working on a patch for this to use a live tree for module ancestry.
I had already implemented this as part of Module#prepend before coming
across this discussion. I wrote a test for this behavior as a starting
point but it fails under the current behavior.

~~~
    m = Module.new
    m.module_eval do
      def foo; :foo; end
    end
    class << m; self; end.class_eval do
      define_method(:included) {|mod| raise }
    end

    m2 = Module.new
    assert_raise(RuntimeError) do
      m2.module_eval { include(m) }
    end

    m3 = Module.new
    assert_raise(RuntimeError) do
      m3.module_eval { include(m2) }
    end
~~~

`m.included` is only called on the include for `m2` not `m3`.

Other than wrapping methods, which would be seen by includes further
down the chain in either the current or live ancestries, I can't think
of an example of hierarchy changes that would fire when including `m2`
into `m3`. Do you have an example of one such action?


----------------------------------------
Feature #9112: Make module lookup more dynamic (Including modules into a
module after it has already been included)
https://bugs.ruby-lang.org/issues/9112#change-49258

* Author: Tobias Pfeiffer
* Status: Assigned
* Priority: Normal
* Assignee: Yukihiro Matsumoto
* Category: core
* Target version:
----------------------------------------
If a module (M) is included into a class (C) and afterwards another
module (M2) is included into the first module (M) then C does not
include M2 and instances do not respond to methods defined in M2. I
think instances of C should respond to methods defined in M2 and C
should include M2.

I created a gist detailing the problem I have:
https://gist.github.com/PragTob/7472643

I think this behavior is confusing, because if I'd reopen module M and
just add methods there then instances of C can call those methods.
However if I include another module in M then instances of C can not
call those methods.

Any opinions on if this would be a better behavior or why it isn't?

(was unsure to file it as a bug or feature)
24d2f8804e6bb4b7ea6bd11e0a586470?d=identicon&s=25 unknown (Guest)
on 2014-10-07 20:38
(Received via mailing list)
Issue #9112 has been updated by Jeremy Kemper.


See #1586

~~~
1. allow multiple inclusion
1. propagate when a module includes a module

Both have been (basically) accepted
~~~

----------------------------------------
Feature #9112: Make module lookup more dynamic (Including modules into a
module after it has already been included)
https://bugs.ruby-lang.org/issues/9112#change-49260

* Author: Tobias Pfeiffer
* Status: Assigned
* Priority: Normal
* Assignee: Yukihiro Matsumoto
* Category: core
* Target version:
----------------------------------------
If a module (M) is included into a class (C) and afterwards another
module (M2) is included into the first module (M) then C does not
include M2 and instances do not respond to methods defined in M2. I
think instances of C should respond to methods defined in M2 and C
should include M2.

I created a gist detailing the problem I have:
https://gist.github.com/PragTob/7472643

I think this behavior is confusing, because if I'd reopen module M and
just add methods there then instances of C can call those methods.
However if I include another module in M then instances of C can not
call those methods.

Any opinions on if this would be a better behavior or why it isn't?

(was unsure to file it as a bug or feature)
87bf8ebf42753b77da12f70e9d1e5671?d=identicon&s=25 unknown (Guest)
on 2014-10-07 20:49
(Received via mailing list)
Issue #9112 has been updated by Timur Duehr.


I understand 1. I'm missing something on 2. Does that mean my example
code above should raise on including `m2`?

----------------------------------------
Feature #9112: Make module lookup more dynamic (Including modules into a
module after it has already been included)
https://bugs.ruby-lang.org/issues/9112#change-49261

* Author: Tobias Pfeiffer
* Status: Assigned
* Priority: Normal
* Assignee: Yukihiro Matsumoto
* Category: core
* Target version:
----------------------------------------
If a module (M) is included into a class (C) and afterwards another
module (M2) is included into the first module (M) then C does not
include M2 and instances do not respond to methods defined in M2. I
think instances of C should respond to methods defined in M2 and C
should include M2.

I created a gist detailing the problem I have:
https://gist.github.com/PragTob/7472643

I think this behavior is confusing, because if I'd reopen module M and
just add methods there then instances of C can call those methods.
However if I include another module in M then instances of C can not
call those methods.

Any opinions on if this would be a better behavior or why it isn't?

(was unsure to file it as a bug or feature)
F1d6cc2b735bfd82c8773172da2aeab9?d=identicon&s=25 Nobuyoshi Nakada (nobu)
on 2014-10-08 05:06
(Received via mailing list)
Issue #9112 has been updated by Nobuyoshi Nakada.


No.
`Module#include` doesn't change superclass of the receiver's metaclass.

----------------------------------------
Feature #9112: Make module lookup more dynamic (Including modules into a
module after it has already been included)
https://bugs.ruby-lang.org/issues/9112#change-49279

* Author: Tobias Pfeiffer
* Status: Assigned
* Priority: Normal
* Assignee: Yukihiro Matsumoto
* Category: core
* Target version:
----------------------------------------
If a module (M) is included into a class (C) and afterwards another
module (M2) is included into the first module (M) then C does not
include M2 and instances do not respond to methods defined in M2. I
think instances of C should respond to methods defined in M2 and C
should include M2.

I created a gist detailing the problem I have:
https://gist.github.com/PragTob/7472643

I think this behavior is confusing, because if I'd reopen module M and
just add methods there then instances of C can call those methods.
However if I include another module in M then instances of C can not
call those methods.

Any opinions on if this would be a better behavior or why it isn't?

(was unsure to file it as a bug or feature)
87bf8ebf42753b77da12f70e9d1e5671?d=identicon&s=25 unknown (Guest)
on 2014-10-08 05:40
(Received via mailing list)
Issue #9112 has been updated by Timur Duehr.


Are you just looking for the following test to pass then for 2?

~~~
  def test_simple_include
    obj = Object.new
    class << obj
      class A; def foo; 1; end; end
      module B; end
      class C < A; include B; def bar; foo; end; end
      $test_simple_include1 = C.new.bar

      module X; def foo; 2; end; end
      module B; include X; end
      $test_simple_include2 = C.new.bar
      module X; def foo; 3; end; end
      $test_simple_include3 = C.new.bar

      module B; def foo; 4; end; end
      $test_simple_include4 = C.new.bar
    end

    assert_equal(1, $test_simple_include1)
    assert_equal(2, $test_simple_include2)
    assert_equal(3, $test_simple_include3)
    assert_equal(4, $test_simple_include4)
  end
~~~

----------------------------------------
Feature #9112: Make module lookup more dynamic (Including modules into a
module after it has already been included)
https://bugs.ruby-lang.org/issues/9112#change-49280

* Author: Tobias Pfeiffer
* Status: Assigned
* Priority: Normal
* Assignee: Yukihiro Matsumoto
* Category: core
* Target version:
----------------------------------------
If a module (M) is included into a class (C) and afterwards another
module (M2) is included into the first module (M) then C does not
include M2 and instances do not respond to methods defined in M2. I
think instances of C should respond to methods defined in M2 and C
should include M2.

I created a gist detailing the problem I have:
https://gist.github.com/PragTob/7472643

I think this behavior is confusing, because if I'd reopen module M and
just add methods there then instances of C can call those methods.
However if I include another module in M then instances of C can not
call those methods.

Any opinions on if this would be a better behavior or why it isn't?

(was unsure to file it as a bug or feature)
0ec4920185b657a03edf01fff96b4e9b?d=identicon&s=25 unknown (Guest)
on 2014-10-08 05:58
(Received via mailing list)
Issue #9112 has been updated by Yukihiro Matsumoto.

Related to Feature #1586: Including a module already present in
ancestors should not be ignored added

----------------------------------------
Feature #9112: Make module lookup more dynamic (Including modules into a
module after it has already been included)
https://bugs.ruby-lang.org/issues/9112#change-49281

* Author: Tobias Pfeiffer
* Status: Assigned
* Priority: Normal
* Assignee: Yukihiro Matsumoto
* Category: core
* Target version:
----------------------------------------
If a module (M) is included into a class (C) and afterwards another
module (M2) is included into the first module (M) then C does not
include M2 and instances do not respond to methods defined in M2. I
think instances of C should respond to methods defined in M2 and C
should include M2.

I created a gist detailing the problem I have:
https://gist.github.com/PragTob/7472643

I think this behavior is confusing, because if I'd reopen module M and
just add methods there then instances of C can call those methods.
However if I include another module in M then instances of C can not
call those methods.

Any opinions on if this would be a better behavior or why it isn't?

(was unsure to file it as a bug or feature)
0ec4920185b657a03edf01fff96b4e9b?d=identicon&s=25 unknown (Guest)
on 2014-10-08 06:02
(Received via mailing list)
Issue #9112 has been updated by Yukihiro Matsumoto.


The reasons behind the current behavior are as you have guessed. Besides
that, it was quite difficult to implement this more dynamic inclusion
efficiently (both space-wise and time-wise).

So with smarter implementation, with proper timing, I don't reject. But
it will not happen in coming 2.2, at least.

Matz.


----------------------------------------
Feature #9112: Make module lookup more dynamic (Including modules into a
module after it has already been included)
https://bugs.ruby-lang.org/issues/9112#change-49283

* Author: Tobias Pfeiffer
* Status: Assigned
* Priority: Normal
* Assignee: Yukihiro Matsumoto
* Category: core
* Target version:
----------------------------------------
If a module (M) is included into a class (C) and afterwards another
module (M2) is included into the first module (M) then C does not
include M2 and instances do not respond to methods defined in M2. I
think instances of C should respond to methods defined in M2 and C
should include M2.

I created a gist detailing the problem I have:
https://gist.github.com/PragTob/7472643

I think this behavior is confusing, because if I'd reopen module M and
just add methods there then instances of C can call those methods.
However if I include another module in M then instances of C can not
call those methods.

Any opinions on if this would be a better behavior or why it isn't?

(was unsure to file it as a bug or feature)
87bf8ebf42753b77da12f70e9d1e5671?d=identicon&s=25 unknown (Guest)
on 2014-10-08 18:12
(Received via mailing list)
Issue #9112 has been updated by Timur Duehr.


Thanks for the input. I think that's enough that I can start working on
it.

Porting the implementation I have for JRuby would yield O(N) method
search time with recursion up to the deepest include/prepend. I'm pretty
sure the current objects will not need modification. Worst case,
`RClass_ext` will need an additional array/linked list. Handling `super`
calls gets a bit trickier. Since the current call location will no
longer be directly in line with `self`'s class, the next superclass up
from a leaf node in the graph will need to be searched for. Fortunately,
this implementation also reduces the number of `T_ICLASS` objects
required since only one is needed every time an include is performed.

This search can also take an iterative approach trading call stack for a
list on the stack. There's a way to do away with that list by linking
the leaf nodes (included modules) to the next/previous node in the
search but that'll add complexity to `include`.

My intent is to start with an iterative BFS since that should be a good
tradeoff between complexity and size, and it shouldn't be too hard to
extend to a threaded BFS search.

Does that sound like a good approach?


----------------------------------------
Feature #9112: Make module lookup more dynamic (Including modules into a
module after it has already been included)
https://bugs.ruby-lang.org/issues/9112#change-49298

* Author: Tobias Pfeiffer
* Status: Assigned
* Priority: Normal
* Assignee: Yukihiro Matsumoto
* Category: core
* Target version:
----------------------------------------
If a module (M) is included into a class (C) and afterwards another
module (M2) is included into the first module (M) then C does not
include M2 and instances do not respond to methods defined in M2. I
think instances of C should respond to methods defined in M2 and C
should include M2.

I created a gist detailing the problem I have:
https://gist.github.com/PragTob/7472643

I think this behavior is confusing, because if I'd reopen module M and
just add methods there then instances of C can call those methods.
However if I include another module in M then instances of C can not
call those methods.

Any opinions on if this would be a better behavior or why it isn't?

(was unsure to file it as a bug or feature)
This topic is locked and can not be replied to.