Ruby Forum Ruby > When introspection doesn't...

Ruby pop quiz.  How can it be that:

  foo.bar()

is different from:

  foo.method(:bar).call()

Is it possible to override the "method" method?
Can you also override the "class" method?

  if foo.class == Array
    Array.instance_method(:find_all).bind(foo)
  end

Results in
  TypeError: bind argument must be an instance of Array

Can you override all of the introspection methods so that there is NO
way to inspect an object and really tell what it is?  If so, then how
did "bind" know the difference?  There has to be a back door, right?

I found some "evil" at:

http://eigenclass.org/hiki.rb?class+hierarchy+introspection+evil.rb


Is this the best way in Ruby?  Python reserved methods with two
underscores that can't be redefined.  Does Ruby have an equivalent?


From the FAQ:
-------------

7.1 How does Ruby choose which method to invoke?

Ruby binds all messages to methods dynamically. It searches first for
singleton methods in the receiver, then for methods defined in the
receiver's own class, and finally for methods defined in the receiver's
superclasses (including any modules which may have been mixed in). You
can see the order of searching by displaying Classname.ancestors, which
shows the ancestor classes and modules of ClassName.

-----------
If this is true, then I only need to check three places to see what
method will be called.  Is there a way for me to look the same way Ruby
does?

For those that are curious.... the real life example that raised this
question for me was in rails.  When you have a Model with a has_many
relationship it creates an Array-like object that looks very much like
an Array, but behaves as described above (with a secret undocumented
deprecated find_all) method.

Hi --

On 3/14/07, Chad Lester <chad.lester@gmail.com> wrote:
>
> Ruby pop quiz.  How can it be that:
>
>   foo.bar()
>
> is different from:
>
>   foo.method(:bar).call()

Here's one time:

irb(main):001:0> class C; def method_missing(m); puts "caught!"; end; 
end
=> nil
irb(main):002:0> C.new.x
caught!
=> nil
irb(main):003:0> C.new.method(:x).call
NameError: undefined method `x' for class `C'

> Is it possible to override the "method" method?
> Can you also override the "class" method?

Sure.  I don't think there are any non-overridable methods in Ruby.
It's all open.

> shows the ancestor classes and modules of ClassName.
>
> -----------
> If this is true, then I only need to check three places to see what
> method will be called.  Is there a way for me to look the same way Ruby
> does?

You can check the various classes and modules with instance_methods
and other similar methods.

> For those that are curious.... the real life example that raised this
> question for me was in rails.  When you have a Model with a has_many
> relationship it creates an Array-like object that looks very much like
> an Array, but behaves as described above (with a secret undocumented
> deprecated find_all) method.

You sound bitter :-)  Keep in mind that class isn't really very
important in Ruby.  Generally you're more concerned with what a given
object actually does.  So while these AR collections report themselves
as Arrays, the main thing is their behavior, which is kind of a
souped-up Array behavior.


David

David,

Thank you for your reply.  It wasn't what I was hoping for, but I still 
appreciate it.

What I meant to ask is when can calling foo.bar() result in a different 
method call from foo.method(:bar).call() in the case where there really 
is a method called 'bar'.  Kudos for pointing out that the error message 
is different when 'bar' doesn't exist, but that wasn't what I was 
looking for. =)

And similarly, why could binding an object to a method fail with a type 
error, even when the object's class method is reporting that it is of 
the correct class.

Yes, it's possible that the object is overriding both "class" and 
"method", but I really don't think so.  And even if you did, it doesn't 
explain how "bind" figures out that it's not really an Array.

I agree that one shouldn't be so worried about what class an object 
reports itself as.  But this is the very reason why I don't think the 
rails implementation would bother overriding "class" and "method".

I'd love to know what mechanism they used to achieve this magic.  If it 
was me, I would have probably made a sub-class of Array.  If not that, 
then I would have inserted my own find_all with an "extend" call.  But 
that would also show up when you called method(:find_all).  So they're 
doing something else.  If push comes to shove, I can start pouring 
through the rails code... but ugh..

I'm sorry if I sounded bitter.  I'm not really.  These things happen. 
There's a  reason why this rails "find_all" method is marked as 
deprecated.  When I was searching for answers I found all kinds of 
confusion it generated (many on this forum), including a patch that was 
submitted a year ago that would have tried to eliminate some of the 
confusion:

http://dev.rubyonrails.org/ticket/2898

But what I'm more interested in is what mechanism in the Ruby language 
is being used to get the alternate find_all behaviour.

Perhaps this would have been a better post for the Rails group, but it 
seemed more like a Ruby language question to me.

So my real question is what really happens when the Ruby interpreter 
see's foo.bar  ???

I'm guessing the answer in the FAQ about how Ruby chooses what method to 
invoke is not really precise.  And I don't think the answer is in the 
common Ruby books.  ;)

Thanks again!
Chad


David A. Black wrote:
> Hi --
> 
> On 3/14/07, Chad Lester <chad.lester@gmail.com> wrote:
>>
>> Ruby pop quiz.  How can it be that:
>>
>>   foo.bar()
>>
>> is different from:
>>
>>   foo.method(:bar).call()
> 
> Here's one time:
> 
> irb(main):001:0> class C; def method_missing(m); puts "caught!"; end; 
> end
> => nil
> irb(main):002:0> C.new.x
> caught!
> => nil
> irb(main):003:0> C.new.method(:x).call
> NameError: undefined method `x' for class `C'
> 
>> Is it possible to override the "method" method?
>> Can you also override the "class" method?
> 
> Sure.  I don't think there are any non-overridable methods in Ruby.
> It's all open.
> 
>> shows the ancestor classes and modules of ClassName.
>>
>> -----------
>> If this is true, then I only need to check three places to see what
>> method will be called.  Is there a way for me to look the same way Ruby
>> does?
> 
> You can check the various classes and modules with instance_methods
> and other similar methods.
> 
>> For those that are curious.... the real life example that raised this
>> question for me was in rails.  When you have a Model with a has_many
>> relationship it creates an Array-like object that looks very much like
>> an Array, but behaves as described above (with a secret undocumented
>> deprecated find_all) method.
> 
> You sound bitter :-)  Keep in mind that class isn't really very
> important in Ruby.  Generally you're more concerned with what a given
> object actually does.  So while these AR collections report themselves
> as Arrays, the main thing is their behavior, which is kind of a
> souped-up Array behavior.
> 
> 
> David

On Mar 14, 2007, at 11:04 PM, Chad Lester wrote:
> is different when 'bar' doesn't exist, but that wasn't what I was
> looking for. =)

This "might" explain it:

It is possible to use method_missing to catch a call to an undefined
method, dynamically create that method, and invoke it.
So foo.bar() would create the method the first time it is called such
that subsequent foo.bar() calls or foo.method(:bar) calls succeed
without any intervention by method_missing.

I believe that this technique is used in Rails but I don't know if it
explains the particular behavior you are seeing.


Gary Wright

Ah, the magic of AssociationProxy.  I ran into weirdness with this
when I was doing a case/when on the return value of a has_many in one
of my models.

On Mar 15, 11:04 am, Chad Lester <chad.les...@gmail.com> wrote:
> And similarly, why could binding an object to a method fail with a type
> error, even when the object's class method is reporting that it is of
> the correct class.
>
> Yes, it's possible that the object is overriding both "class" and
> "method", but I really don't think so.  And even if you did, it doesn't
> explain how "bind" figures out that it's not really an Array.

Looking at the C source for bind(), it checks the object's underlying
class against the receiving class, and does so without going through
any of the re-defined methods for the given object.  This allows it to
peek into the real class for the given object even if the object over-
rode Object#class.

AFAIK, there's no way to prevent bind() from doing this.

> I'd love to know what mechanism they used to achieve this magic.  If it
> was me, I would have probably made a sub-class of Array.  If not that,
> then I would have inserted my own find_all with an "extend" call.  But
> that would also show up when you called method(:find_all).  So they're
> doing something else.  If push comes to shove, I can start pouring
> through the rails code... but ugh..

It actually, implicitly, does re-define both of these methods.  Take a
look at lib/active_record/associations/association_proxy.rb

ActiveRecord associations are all implemented using a proxy of a real
array as returned by find().  The proxy undefs *all* instance methods
except for those that begin with underscores among some other
'important' methods.  It then defines a method_missing method which
catches any undefined method references and passes them to the
underlying array.

If the association object defines any method that shares the same name
as the underlying array object, it'll be called instead of
method_missing which prevents the association proxy from handing it to
the underlying array.

So, in your case, the HasManyAssociation defines #find_all which
causes it to be called instead of method_missing...

AssociationProxy defines #target which allows you to retrieve the
underlying object:

>> Array.instance_method(:find_all).bind(foo.target)
=> #<Method: Array(Enumerable)#find_all>

Eden,

Thanks a million.  You've definitely solved this mystery for me.  I 
wouldn't have thought that they would explicitly redefined the 'class' 
method, but they did because it was just the easiest thing to do...  I 
gave myself a little homework assignment and wrote a very small program 
that demonstrates how easy it is to do this.

I learned about undef_method and method_missing (which probably should 
be in the FAQ about how Ruby decides which method gets called).

Hey, you should get compensated for this.  I'll send ya 50 bucks if you 
tell me how.  :)  same user name at gmail.

Also... it does kind of make wish there were some methods that one could 
not override, just so you can poke in and see what's going on without 
resorting to c programming.  Or maybe this would be better done with a 
special inspector object that knows how to inspect objects... I'll look 
for one, and then maybe write one if I can't find anything.


Here's my little example, based on Eden's help:
---------------------------

class Proxy
  instance_methods.each { |m|
     undef_method m unless m =~ /(^__|^nil\?$|^send$|proxy_)/
  }
  def initialize(target)
    @target = target
  end
  def method_missing(method, *args, &block)
    # most messages just get routed to the target
    @target.send(method, *args, &block)
  end
  def find_all(*args, &block)
    # super-secret find_all that completely ignores block!
    @target
  end
end

a = [1,2,3,4,5,6]
p = Proxy.new(a)

puts p.class
puts p.method(:find_all)

my_block = Proc.new { |x| x % 2 == 0 }

puts (p.find_all &my_block).inspect
puts (p.method(:find_all).call &my_block).inspect

if p.class == Array
  Array.instance_method(:find_all).bind(p)
end


Output looks like:

Array
#<Method: Array(Enumerable)#find_all>
[1, 2, 3, 4, 5, 6]
[2, 4, 6]
./proxy_fun.rb:32:in `bind': bind argument must be an instance of Array 
(TypeError)
        from ./proxy_fun.rb:32

The power of Ruby :)  Everything can be redefined or undefined,
although there are a few warnings that Ruby will complain about:

$ irb
irb(main):001:0> class C; undef_method :__send__, :__id__; end
(irb):1: warning: undefining `__send__' may cause serious problem
(irb):1: warning: undefining `__id__' may cause serious problem

You can also look at remove_method which will get rid of any methods
defined in your class after you've defined them, but it won't touch
anything in the super class.

You might have to resort to C to write the "super introspector" since
some of the internal data structures ruby uses aren't accessible from
the language itself (or at least that seems to be the case from the
cursory inspection I did on eval.c).

For instance -- I couldn't find any way to get the receiving class
from Method or UnboundMethod... they're locked up in struct METHOD
somehow.  The only hack I could come up with was to parse the output
of to_s:

class UnboundMethod
  def rclass
    to_s.scan(/: (\w+)[(#]/).flatten.map{|s|Object.const_get(s)}.first
  end
end

>> Array.instance_method(:[]).class
=> UnboundMethod
>> Array.instance_method(:[]).rclass
=> Array