Re-implementing Autoload

aris · January 9, 2013, 12:01am

Finally accepting the fact that autoload will never be fixed (i.e. there
it
no way to override require to effect autoload’s behavior) I decided to
try re-implementing autoload in Ruby itself.

So here’s what I arrived at:

https://github.com/rubyworks/autoload/blob/master/lib/autoload.rb

I thought that was it, but then I tried a bundle exec call with this
autoload.rb loaded (via RUBYOPT), and it bombs.

autoload.rb:104:in `const_missing': uninitialized constant

Module::Settings (NameError)
from
/home/trans/.gem/ruby/1.9.3/gems/bundler-1.2.3/lib/bundler.rb:191:in
settings' from /home/trans/.gem/ruby/1.9.3/gems/bundler-1.2.3/lib/bundler.rb:330:inconfigure_gem_home_and_path’
from /home/trans/.gem/ruby/1.9.3/gems/bundler-1.2.3/lib/bundler.rb:86:in
configure' from /home/trans/.gem/ruby/1.9.3/gems/bundler-1.2.3/lib/bundler.rb:142:indefinition’
from
/home/trans/.gem/ruby/1.9.3/gems/bundler-1.2.3/lib/bundler/cli.rb:423:in
exec' from /home/trans/.gem/ruby/1.9.3/gems/bundler-1.2.3/lib/bundler/vendor/thor/task.rb:27:inrun’
from
/home/trans/.gem/ruby/1.9.3/gems/bundler-1.2.3/lib/bundler/vendor/thor/invocation.rb:120:in
invoke_task' from /home/trans/.gem/ruby/1.9.3/gems/bundler-1.2.3/lib/bundler/vendor/thor.rb:275:indispatch’
from
/home/trans/.gem/ruby/1.9.3/gems/bundler-1.2.3/lib/bundler/vendor/thor/base.rb:408:in
start' from /home/trans/.gem/ruby/1.9.3/gems/bundler-1.2.3/bin/bundle:14:inblock
in <top (required)>’
from
/home/trans/.gem/ruby/1.9.3/gems/bundler-1.2.3/lib/bundler/friendly_errors.rb:4:in
with_friendly_errors' from /home/trans/.gem/ruby/1.9.3/gems/bundler-1.2.3/bin/bundle:14:in<top
(required)>’
from /home/trans/.gem/ruby/1.9.3/bin/bundle:23:in `’

Turns out that the self in const_missing that causes this is
#<Class:Bundler>. How can I workout the Bundler namespace given
that?

All other advice on improving this is also appreciated.

tcsnide · January 9, 2013, 2:56am

I was able to implement a hack:

# if module has no name, try to parse out a namespace from #insepct.
# (yes, this is a hack!)
unless parent
  if name.nil?
    if /#<Class:(.*?)>/ =~ self.inspect
      parent = Object.const_get($1) rescue nil
    end
  end
end

That seems to do the trick for loading, but for some reason I get an
even
odder error:

/home/trans/.gem/ruby/1.9.3/gems/bundler-1.2.3/lib/bundler/dsl.rb:128:in
`initialize’: wrong number of arguments(1 for 0) (ArgumentError)

/home/trans/.gem/ruby/1.9.3/gems/bundler-1.2.3/lib/bundler/dsl.rb:128:in
`new’

/home/trans/.gem/ruby/1.9.3/gems/bundler-1.2.3/lib/bundler/dsl.rb:128:in
`path’

/home/trans/.gem/ruby/1.9.3/gems/bundler-1.2.3/lib/bundler/dsl.rb:244:in
`block in _normalize_options’

/home/trans/.gem/ruby/1.9.3/gems/bundler-1.2.3/lib/bundler/dsl.rb:237:in
`each’

/home/trans/.gem/ruby/1.9.3/gems/bundler-1.2.3/lib/bundler/dsl.rb:237:in
`_normalize_options’

/home/trans/.gem/ruby/1.9.3/gems/bundler-1.2.3/lib/bundler/dsl.rb:70:in
`gem’

/home/trans/.gem/ruby/1.9.3/gems/bundler-1.2.3/lib/bundler/dsl.rb:49:in
gemspec' /home/rubyworks/Projects/all/autoload/Gemfile:2:ineval_gemfile’

Note the Gemfile is jsut the typical source :rubygems; gemspec.

tcsnide · January 9, 2013, 9:43am

You surely know that, but let me comment that this autoload behaves in
a different way.

You sure know the original autoload is transparently hooked into each
step of constant resolution and does not go through const_missing.
Also, since const_missing does not know the nesting, and does not know
the resolution algorithm that is being used, you just cannot emulate
it.

Active Support autoloading suffers from the same constraints and that
is why it does not claim to emulate constant resolution (it cannot),
rather AS autoloading gives you a series of conventions that if you
follow then it works. You have to program to a contract.

tcsnide · January 10, 2013, 2:21pm

Thanks Xavier. I realize implementation differs. Clearly I can’t tap
into
the underlying constant resolution with pure Ruby. But I expected that I
should be able to emulate it effectively with const_missing --I would
think
it would possible to get essentially the same results.

The only difference I am sure about at this point, is that I haven’t
been
able to find a way to get Module.nesting, instead I had to resort to
Constant#name and splitting it up. Yet I thought this would give me a
superset of current functionality, not a subset. i.e.

class Foo
  HERE = "here"
  class Bar::Baz
    HERE
  end
end

Ruby’s constant lookup would not find HERE, my code would.

Obviously, I am still missing something. And maybe you are right, that
it
is not possible (which would be sad, imo). I just wish I knew what it
was.

tcsnide · January 10, 2013, 4:51pm

On Thursday, January 10, 2013 8:55:17 AM UTC-5, Xavier N. wrote:

  X
end
there is a last step that manually checks Object in spite of Object
not being an ancestor of M, neither belonging to the nesting.

Ah, thanks. I’ve incorporated that into my implementation. I assume it’s
just looks in Object in this case and not all it’s ancestors [Object,
Kernel, BasicObject]?

As far as I know, those are the two existing limitations for emulating
constant name resolution within const_missing.

But does this explain the error I get with bundle exec? That’s what I
don’t understand still. I don’t think this one difference accounts for
it.
Maybe it does, but I don’t see how. And if it does not, then there
remains
something else amiss.

tcsnide · January 10, 2013, 2:56pm

On Thu, Jan 10, 2013 at 2:08 PM, Intransition [email protected]
wrote:

Thanks Xavier. I realize implementation differs. Clearly I can’t tap into
the underlying constant resolution with pure Ruby. But I expected that I
should be able to emulate it effectively with const_missing --I would think
it would possible to get essentially the same results.

The only difference I am sure about at this point, is that I haven’t been
able to find a way to get Module.nesting, instead I had to resort to
Constant#name and splitting it up.

Exactly, dependencies.rb does the same as a know trade-off.

The second bit const_missing lacks is the resolution algorithm that
failed to find the constant.

Point is, while resolving X in

module M
  X
end

there is a last step that manually checks Object in spite of Object
not being an ancestor of M, neither belonging to the nesting.

In

M::X

this manual lookup is not performed.

Thus, in one case you should be able to resolve an autoloded X, and in
the second one you should not, and you are not told which is the
case.

As far as I know, those are the two existing limitations for emulating
constant name resolution within const_missing.

This is a pity, I would love to rewrite dependencies.rb entirely to
base it in Kernel#autoload and thus removing all these deviations from
the standard way things work, but there are a few blockers. The ones
that motivate your exploration of this topic I am certain. Should
write a post someday about these blockers.