Autoload with a block?

Ruby
1

I’ve just discovered Ruby’s autoload feature. It looks to imply
something like
this:

autoload :Foo, ‘foo’
Bar.autoload :Baz, ‘bar/baz’

This implies that to have namespace’d modules autoloaded properly, I
would end
up with a structure like this:

autoload :Foo, ‘foo’

(and, in foo.rb)

module Foo
autoload :Bar, ‘foo/bar’
end

What if I want to abstract away such a system? In other words, what if I
want
a light, portable version of Rails-style autoloading? In a perfect
world, I
would be able to do something like this:

autoload :Foo do
require ‘foo’
Foo.autoload :Bar …
end

In that simplest form, it would at least centralize all my loading stuff
in
one place. In a more meta form, I could run through some search paths
I’m
likely to use (in my own app, for instance), and generate an autoload
scheme
based on that.

Does this sound like a good idea? Or should I be hacking it together
with
const_missing, instead?

2

No one had anything to say about this… Hmm.

In case you didn’t realize: I’m testing all of this in Ruby 1.9.

I thought of a sneakier approach: The rdoc specifies that autoload will
call
Kernel::require:

http://ruby-doc.org/core-1.9/classes/Kernel.html#M006100

And it does, indeed, seem to behave this way – so long as I don’t
redefine
Kernel::require. A simple test:

$ irb1.9
irb(main):001:0> module Kernel
irb(main):002:1> alias_method :require_without_me, :require
irb(main):003:1> def require(*args, &block)
irb(main):004:2> puts ‘Require called:’
irb(main):005:2> p args
irb(main):006:2> p block
irb(main):007:2> require_without_me *args, &block
irb(main):008:2> end
irb(main):009:1> end
=> nil
irb(main):010:0> autoload :Foo, ‘foo’
=> nil
irb(main):011:0> Foo
loaded foo.rb
=> Foo
irb(main):012:0> require ‘foo/bar’
Require called:
[“foo/bar”]
nil
loaded bar.rb
=> true
irb(main):013:0>

For what it’s worth, both foo.rb and foo/bar.rb have ‘puts’ statements
in
them, declaring that they were loaded.

Maybe it’s irb at fault? Hmm… I created this file:

module Kernel
alias_method :require_without_me, :require
def require(*args, &block)
if args.first == ‘test/file/not/loading’
puts ‘Not loading the test file’
else
require_without_me(*args, &block)
end
end
end

puts ‘The following works:’
require ‘test/file/not/loading’

puts ‘The following does not:’
autoload :Foo, ‘test/file/not/loading’
Foo

I’d expect it to end with a constant not found, or module not found.
Instead, I get:

$ ruby1.9 test.rb
The following works:
Not loading the test file
The following does not:
test.rb:17:in `': no such file to load – test/file/not/loading
(LoadError)

Doesn’t look like I typoed.

So, in other words, autoload is either using its own interpretation of
require, or it’s hardcoding (in C) a call to the original require,
rather
than going through the Ruby code. (I wonder how this would have
cooperated
with Rubygems, had that not been included?)

So far, the crushing limitation of autoload is that I can’t easily
autoload
namespace’d things, without littering the namespace itself with autoload
methods – or eager-loading all parent namespaces.

Well, that, and I can’t seem to customize its behavior at_all other
than by
rewriting it from scratch, making it somewhat useless.

3

On Aug 16, 1:18 am, David M. [email protected] wrote:

So, in other words, autoload is either using its own interpretation of
require, or it’s hardcoding (in C) a call to the original require, rather
than going through the Ruby code. (I wonder how this would have cooperated
with Rubygems, had that not been included?)

I reported this bug a long time ago. It is a serious issue with one of
my programs. Because of it I am forced never to use autoload, and tell
all users of my program that they must do the same. I would have
expected this to be fixed for 1.9, and back-ported. It’s disappointing
to hear it still remains an issue.

T.

4

On Saturday 16 August 2008 07:33:22 Trans wrote:

It is a serious issue with one of
my programs. Because of it I am forced never to use autoload, and tell
all users of my program that they must do the same.

I think I’ve found a solution, for myself – basically a const_missing
hack.

Actually, I cracked open the ActiveSupport source, curious to see how
they did
it. And I spent probably an hour or two, staring at the code, before
shaking
my head and starting from scratch.

Rails lets you write beautiful code, sometimes, but the Rails source
sure
isn’t beautiful, most of the time. (In my not-so-humble opinion.)

Still, I can’t help but wonder – if ActiveSupport had to do such
backflips to
get it working, maybe there’s a serious assumption in my own code?

Let me know if I’m missing something.
I’m still trying to think of ways this could break.

You can use anything that provides an inflector.

I’m already using Sequel here, so it works for me.

require ‘sequel’

class Module
def const_missing_with_autoload(name)
our_name = (self==Object) ? ‘’ : self.name + ‘::’
begin
require (our_name+name.to_s).underscore
const_get name
rescue LoadError
const_missing_without_autoload(name)
end
end
alias_method :const_missing_without_autoload, :const_missing
alias_method :const_missing, :const_missing_with_autoload
end

5

On Aug 16, 4:28 pm, David M. [email protected] wrote:

Or am I really going to have to create a temporary file for each and every
source file I might possibly ever want to use?

The things to do is of course to fix the Ruby source to A) use a
callback for autoload and B) to use the normal Kernel.require call.
Then submit a patch and pray the higher ups care enough to check it
in.

Now the question is are you a C coder :wink:

T.

6

On Saturday 16 August 2008 12:55:42 David M. wrote:

On Saturday 16 August 2008 07:33:22 Trans wrote:

It is a serious issue with one of
my programs. Because of it I am forced never to use autoload, and tell
all users of my program that they must do the same.

I think I’ve found a solution, for myself – basically a const_missing hack.

Ok, that didn’t work. All kinds of edge cases, and I can see why things
like
autoload have been written.

So I wrote my own… but there’s still one problem I can’t seem to solve
here:

class AutoLoader < BasicObject
def initialize parent, mod, file
@parent, @mod, @file = parent, mod, file
end
end

Pushed out here so I can get at Kernel.require.

AutoLoader.send :define_method, :method_missing do |*args, &block|
@parent.send :remove_const, @mod
Kernel.require @file
@parent.const_get(@mod).send *args, &block
end

Typical usage would be:

self.class.const_set :Foo, AutoLoader.new(self.class, :Foo, ‘foo’)

And it works great – exactly the way I want – for almost all cases.
I still can’t do this:

module Foo
end

without actually loading foo.rb yet. (Somehow, Ruby knows my BasicObject
isn’t
a module, without ever tripping the method_missing there.)

Yet the same thing does work with autoload:

autoload :Foo, ‘foo’
module Foo; end

For what it’s worth, Rails’ const_missing has an even worse problem: If
I do
the “module Foo” syntax above, I will define Foo, which means
const_missing
will never be triggered, which means foo.rb will never be loaded.

It looks like my choices are either to learn to write a (non-portable) C
extension (and maybe still not have it work?), or to create a temporary
file,
point autoload at that, and have it do the rest.

That sucks.

Is there any way to know for certain when a file is loaded, when it’s
autoload
that’s doing it, short of modifying the file? (I’m specifically wanting
to be
notified, so that I can prepare sub-constants to be autoloaded as well

that is, when Foo is loaded, I want to automatically run Foo.autoload
:Bar.)

Is there any way to force autoload to use a different mechanism?

Is there any way, in Ruby, to emulate autoload with actual feature
parity?

Or am I really going to have to create a temporary file for each and
every
source file I might possibly ever want to use?

7

Hi,

At Fri, 15 Aug 2008 15:57:21 +0900,
David M. wrote in [ruby-talk:311365]:

autoload :Foo do
require ‘foo’
Foo.autoload :Bar …
end

In that simplest form, it would at least centralize all my loading stuff in
one place. In a more meta form, I could run through some search paths I’m
likely to use (in my own app, for instance), and generate an autoload scheme
based on that.

I don’t think it makes things simpler, but it sounds
interesting. You can file it in the ITS, if you want.
http://redmine.ruby-lang.org/projects/ruby/issues?set_filter=1&tracker_id=2

8

On Saturday 16 August 2008 21:40:32 Trans wrote:

On Aug 16, 4:28 pm, David M. [email protected] wrote:

Or am I really going to have to create a temporary file for each and every
source file I might possibly ever want to use?

Actually, I have figured out something that works reasonably well, for
now.
Given a path that can be safely autoloaded, it will check for files in
there
that look like they could be modules.

It also provides a mixin, which, when included, checks through its
existing
paths for possible nested modules and adds them.

I suspect it’s twice as long as it needs to be, though, and certainly
less
elegant to write.

If anyone’s interested, I can clean it up and release it. It’s roughly
equivalent to the same functionality in ActiveSupport, except that it’s
much
shorter (dependencies.rb is around 500 lines, and I’m well under 100),
and
requires different kinds of babysitting.

That is: With ActiveSupport, if foo.rb is not loaded, the following:

class Foo; end

will cause foo.rb to never be autoloaded, because const_missing isn’t
checked on assigning things to a constant.

However, in my version, to support foo/bar.rb, I have to do at least:

module Foo
include AutoLoad
end

I can put it in foo.rb, or anywhere that’s loaded by the time I need
Foo::Bar,
but it is required.

The things to do is of course to fix the Ruby source to A) use a
callback for autoload and B) to use the normal Kernel.require call.
Then submit a patch and pray the higher ups care enough to check it
in.

I’d add a C) Have const_missing actually called in enough places to make
it
possible to implement vanilla autoload on top of const_missing.

Having autoload be more flexible in the action it performs would be
helpful,
but it’d be even more helpful for it to be flexible in the constants it
applies to.

Now the question is are you a C coder :wink:

Yes and no.

I have written one-line kernel patches, which did things I’m ashamed to
admit.
I’ve cracked open abandoned projects, to try to make them, for example,
compile on 64-bit. I’ve read the source of poorly-documented parts of
PowerDNS to figure out what it was doing.

To be a programmer on any Unix is to have to be at least somewhat aware
of
what C is and what it can do, even if you never use it.

But I have never done any significant development in C. Even if I had,
it
would (I assume) still take some time to learn the Ruby codebase and
conventions.

So, someone else needs to do this :frowning:

9

Issue #462 has been updated by Yukihiro M…

Status changed from Open to Rejected

the code in the block may not be executed (if the constant is defined
anywhere else), so it is not stable.

http://redmine.ruby-lang.org/issues/show/462

10

Hi,

At Mon, 18 Aug 2008 10:35:01 +0900,
Nobuyoshi N. wrote in [ruby-talk:311603]:

based on that.

I don’t think it makes things simpler, but it sounds
interesting. You can file it in the ITS, if you want.
http://redmine.ruby-lang.org/projects/ruby/issues?set_filter=1&tracker_id=2

Although the OP doesn’t seem to intend to request this feature,
I move it to ruby-core since it feels interesting.

Index: load.c

— load.c (revision 18653)
+++ load.c (working copy)
@@ -618,10 +618,29 @@ ruby_init_ext(const char *name, void (*i

static VALUE
-rb_mod_autoload(VALUE mod, VALUE sym, VALUE file)
+rb_mod_autoload(int argc, VALUE *argv, VALUE mod)
{

  • ID id = rb_to_id(sym);
  • Check_SafeStr(file);
  • rb_autoload(mod, id, RSTRING_PTR(file));
  • extern VALUE rb_autoload_proc(VALUE, ID, VALUE, VALUE);
  • ID id;
  • VALUE block = rb_block_given_p() ? rb_block_proc() : Qnil;
  • VALUE arg = Qnil;
  • switch (argc) {
  •  case 2:
  • arg = argv[1];
  • break;
  •  case 1:
  • if (!NIL_P(block)) break;
  •  default:
  • rb_raise(rb_eArgError, “wrong number of arguments (%d for 2)”, argc);
  • }
  • id = rb_to_id(argv[0]);
  • if (NIL_P(block)) {
  • VALUE file = argv[1];
  • Check_SafeStr(file);
  • rb_autoload(mod, id, RSTRING_PTR(file));
  • }
  • else {
  • rb_autoload_proc(mod, id, block, arg);
  • }
    return Qnil;
    }
    @@ -649,5 +668,5 @@ rb_mod_autoload_p(VALUE mod, VALUE sym)

static VALUE
-rb_f_autoload(VALUE obj, VALUE sym, VALUE file)
+rb_f_autoload(int argc, VALUE *argv, VALUE obj)
{
VALUE klass = rb_vm_cbase();
@@ -655,5 +674,5 @@ rb_f_autoload(VALUE obj, VALUE sym, VALU
rb_raise(rb_eTypeError, “Can not set autoload on singleton class”);
}

  • return rb_mod_autoload(klass, sym, file);
  • return rb_mod_autoload(argc, argv, klass);
    }

@@ -693,7 +712,7 @@ Init_load()
rb_define_global_function(“load”, rb_f_load, -1);
rb_define_global_function(“require”, rb_f_require, 1);

  • rb_define_method(rb_cModule, “autoload”, rb_mod_autoload, 2);
  • rb_define_method(rb_cModule, “autoload”, rb_mod_autoload, -1);
    rb_define_method(rb_cModule, “autoload?”, rb_mod_autoload_p, 1);
  • rb_define_global_function(“autoload”, rb_f_autoload, 2);
  • rb_define_global_function(“autoload”, rb_f_autoload, -1);
    rb_define_global_function(“autoload?”, rb_f_autoload_p, 1);

Index: variable.c

— variable.c (revision 18653)
+++ variable.c (working copy)
@@ -1322,16 +1322,12 @@ check_autoload_table(VALUE av)
}

-void
-rb_autoload(VALUE mod, ID id, const char *file)
+typedef VALUE rb_autoload_func(VALUE, ID, VALUE, VALUE);
+
+static void
+autoload_callback(VALUE mod, ID id, rb_autoload_func *func, VALUE arg1,
VALUE arg2)
{

  • VALUE av, fn;
  • VALUE av;
    struct st_table *tbl;
  • if (!rb_is_const_id(id)) {
  • rb_raise(rb_eNameError, “autoload must be constant name: %s”,
    rb_id2name(id));
  • }
  • if (!file || !*file) {
  • rb_raise(rb_eArgError, “empty file name”);
  • }

  • NODE *n;

    if ((tbl = RCLASS_IV_TBL(mod)) && st_lookup(tbl, id, &av) && av !=
    Qundef)
    @@ -1348,8 +1344,61 @@ rb_autoload(VALUE mod, ID id, const char
    DATA_PTR(av) = tbl = st_init_numtable();
    }

  • n = rb_node_newnode(NODE_MEMO, (VALUE)func, arg1, arg2);

  • st_insert(tbl, id, (st_data_t)n);
    +}

+static VALUE
+autoload_load(VALUE klass, ID id, VALUE file, VALUE safe)
+{

  • return rb_require_safe(file, (int)safe);
    +}

+static void
+check_autoload_id(ID id)
+{

  • if (!rb_is_const_id(id)) {
  • rb_raise(rb_eNameError, “autoload must be constant name: %s”,
    rb_id2name(id));
  • }
    +}

+void
+rb_autoload(VALUE mod, ID id, const char *file)
+{

  • VALUE fn;
  • check_autoload_id(id);
  • if (!file || !*file) {
  • rb_raise(rb_eArgError, “empty file name”);
  • }
  • fn = rb_str_new2(file);
    FL_UNSET(fn, FL_TAINT);
    OBJ_FREEZE(fn);
  • st_insert(tbl, id, (st_data_t)rb_node_newnode(NODE_MEMO, fn,
    rb_safe_level(), 0));
  • autoload_callback(mod, id, autoload_load, fn,
    (VALUE)rb_safe_level());
    +}

+void
+rb_autoload_callback(VALUE mod, ID id, rb_autoload_func *func, VALUE
arg1, VALUE arg2)
+{

  • check_autoload_id(id);
  • autoload_callback(mod, id, func, arg1, arg2);
    +}

+static VALUE
+autoload_proc_call(VALUE mod, ID id, VALUE proc, VALUE arg)
+{

  • VALUE args[3];
  • args[0] = mod;
  • args[1] = ID2SYM(id);
  • args[2] = arg;
  • return rb_proc_call_with_block(proc, 3, args, Qnil);
    +}

+void
+rb_autoload_proc(VALUE mod, ID id, VALUE proc, VALUE arg)
+{

  • rb_autoload_callback(mod, id, autoload_proc_call, proc, arg);
    }

@@ -1382,25 +1431,34 @@ VALUE
rb_autoload_load(VALUE klass, ID id)
{

  • VALUE file;
    NODE *load = autoload_delete(klass, id);

  • if (!load || !(file = load->nd_lit)) {

  • if (!load) {
    return Qfalse;
    }
  • return rb_require_safe(file, load->nd_nth);
  • return load->u1.cfunc(klass, id, load->u2.value, load->u3.value);
    }

static VALUE
-autoload_file(VALUE mod, ID id)
+autoload_p(VALUE mod, ID id)
{
VALUE val, file;
struct st_table *tbl;

  • st_data_t load;
  • st_data_t data;
  • NODE *load;
  • if (!st_lookup(RCLASS_IV_TBL(mod), autoload, &val) ||
  • !(tbl = check_autoload_table(val)) || !st_lookup(tbl, id, &load)) {
  • if (!st_lookup(RCLASS_IV_TBL(mod), autoload, &data) ||
  • !(tbl = check_autoload_table(val = (VALUE)data)) ||
  • !st_lookup(tbl, id, &data)) {
    return Qnil;
    }
  • file = ((NODE *)load)->nd_lit;
  • load = (NODE *)data;
  • if (load->u1.cfunc == autoload_proc_call) {
  • /* rb_obj_is_proc() */
  • return load->u2.value;
  • }
  • if (load->u1.cfunc != autoload_load) {
  • return Qtrue;
  • }
  • file = load->u2.value;
    Check_Type(file, T_STRING);
    if (!RSTRING_PTR(file) || !*RSTRING_PTR(file)) {
    @@ -1433,5 +1491,5 @@ rb_autoload_p(VALUE mod, ID id)
    return Qnil;
    }
  • return autoload_file(mod, id);
  • return autoload_p(mod, id);
    }

@@ -1630,5 +1688,5 @@ rb_const_defined_0(VALUE klass, ID id, i
while (tmp) {
if (RCLASS_IV_TBL(tmp) && st_lookup(RCLASS_IV_TBL(tmp), id, &value))
{

  •  if (value == Qundef && NIL_P(autoload_file(klass, id)))

  •  if (value == Qundef && NIL_P(autoload_p(klass, id)))
    return Qfalse;
    return Qtrue;