Kernel#autoload vs require_relative

Ruby
1

I understand that for require to work relative to the given FILE’s
directory one needs to either $LOAD_PATH << File.dirname(FILE) or
use require_relative (with the latter being preferred, I presume).

Is there such an alternative for Kernel#autoload
– and if not, should there be one?

I’m thinking about lib/foo.rb containing

module Foo
autoload :Bar, ‘foo/bar.rb’
autoload :Baz, ‘foo/baz.rb’
autoload :Qux, ‘foo/qux.rb’
end

(which doesn’t work without $LOAD_PATH munging), as
opposed to the (currently working) lib/foo.rb with

require_relative ‘foo/bar.rb’
require_relative ‘foo/baz.rb’
require_relative ‘foo/qux.rb’

— Piotr S.

2

On Jun 16, 2011, at 04:29 , Piotr S. wrote:

autoload :Bar, ‘foo/bar.rb’
autoload :Baz, ‘foo/baz.rb’
autoload :Qux, ‘foo/qux.rb’
end

(which doesnt work without $LOAD_PATH munging),

I wouldn’t call having ‘lib’ in your LOAD_PATH as munging. I’d call that
business as usual. Rubygems supports that as a default. All the usual
rake tasks and test tools support that as a default. It just seems sane
to me… Much more sane than File.dirname(FILE) does or
require_relative.

(btw, you don’t need the “.rb” part–that’s just ugly)

as
opposed to the (currently working) lib/foo.rb with

require_relative ‘foo/bar.rb’
require_relative ‘foo/baz.rb’
require_relative ‘foo/qux.rb’

again, “.rb” not needed.

3

On Jun 16, 7:29am, Piotr S. [email protected] wrote:

autoload :Bar, ‘foo/bar.rb’
autoload :Baz, ‘foo/baz.rb’
autoload :Qux, ‘foo/qux.rb’
end

(which doesnt work without $LOAD_PATH munging), as
opposed to the (currently working) lib/foo.rb with

require_relative ‘foo/bar.rb’
require_relative ‘foo/baz.rb’
require_relative ‘foo/qux.rb’

The question is, are these libs going to loaded anyway regardless of
what is done with the library? If so than you should just go ahead and
load the libs right away with require_relative. Moreover,
require_relative doesn’t have to search though all the load paths, so
it will speed up load times. autoload is only useful for a class or
module that is frequently not used in any given run.

4

Ryan D.:

On Jun 16, 2011, at 04:29 , Piotr S. wrote:

I understand that for require to work relative to the given FILE’s
directory one needs to either $LOAD_PATH << File.dirname(FILE) or
use require_relative (with the latter being preferred, I presume).

Is there such an alternative for Kernel#autoload
– and if not, should there be one?

I’m thinking about lib/foo.rb containing

module Foo
autoload :Bar, ‘foo/bar’
autoload :Baz, ‘foo/baz’
autoload :Qux, ‘foo/qux’
end

(which doesn’t work without $LOAD_PATH munging),

I wouldn’t call having ‘lib’ in your LOAD_PATH as munging.

Ok (and I understand your arguments).

It just seems sane to me… Much more sane than
File.dirname(FILE) does or require_relative.

Hm, I do prefer using require_relative over require + $LOAD_PATH
changes; it looks cleaner to me and doesn’t touch shared
globals, but I guess that’s more of a personal preference.

What I was trying to ask is whether there couldn’t be
a Kernel#autoload_relative call or a ‘relative: true’
optional parameter to Kernel#autoload to make it symmetric
with the require/require_relative combination. I understand
your answer would be ‘there’s no need’. :slight_smile:

(btw, you don’t need the “.rb” part–that’s just ugly)

Fully agreed; fixed in the quotation above
(I shouldn’t post before my morning coffee).

— Piotr S.

5

Intransition:

On Jun 16, 7:29 am, Piotr S. [email protected] wrote:

$LOAD_PATH << File.dirname(FILE)
module Foo
autoload :Bar, ‘foo/bar’
autoload :Baz, ‘foo/baz’
autoload :Qux, ‘foo/qux’
end

require_relative ‘foo/bar’
require_relative ‘foo/baz’
require_relative ‘foo/qux’

The question is, are these libs going to loaded
anyway regardless of what is done with the library?

In my case it depends on the library in question; I was
looking for an approach that could use a relative-path
Kernel#autoload without touching $LOAD_PATH.

— Piotr S.

6

Piotr S.:

Hm, I do prefer using require_relative over require + $LOAD_PATH
changes; it looks cleaner to me and doesn’t touch shared
globals, but I guess that’s more of a personal preference.

…and now I recalled my particular use-case:

require_relative ‘set’

will pull in ‘my’ Set, whereas

require ‘set’

will pull in either ‘my’ Set or the stdlib’s Set depending on whether
I prepended or appended the $LOAD_PATH with File.dirname(FILE).

// I do agree name clashes are at best avoided and at worst
// thoroughly documented, with require ‘./set’ being a (somewhat
// ugly) solution as well, but require_relative’s independence
// from $LOAD_PATH ordering does look appealing to me, hence my
// longing for Kernel#autoload_relative. :slight_smile:

— Piotr S.

7

On Jun 16, 2011, at 10:07 , Piotr S. wrote:

will pull in my Set, whereas

require ‘set’

will pull in either my Set or the stdlibs Set depending on whether
I prepended or appended the $LOAD_PATH with File.dirname(FILE).

I know you know this but it needs to be said for others:

require ‘my/set’

8

Hm, I do prefer using require_relative over require + $LOAD_PATH

changes; it looks cleaner to me and doesnt touch shared

globals, but I guess thats more of a personal preference.

It’s also more secure, which is why . was removed in the first place. :slight_smile:

9

On Thursday, June 16, 2011 11:50:57 AM Piotr S. wrote:

Hm, I do prefer using require_relative over require + $LOAD_PATH
changes; it looks cleaner to me and doesn’t touch shared
globals, but I guess that’s more of a personal preference.

So, while writing the rest of this (long) post, I think I actually dug
up a
solution for you, in the form of the ‘autoloader’ gem I wrote awhile
back:

If everything in your ‘lib’ hierarchy is normally loaded via autoload,
and
follows a decent naming convention, I think this solves your problem. In
your
example, somewhere in ‘lib/foo.rb’, you could do this:

require ‘autoloader’
AutoLoader << File.dirname(FILE)

module Foo
include AutoLoader
end

Ok, there’s some FILE ugliness, but in exchange, you automagically
get
‘autoload’ statements generated for everything, and you don’t have to
modify
your load path.

Caveat: I wrote this awhile ago, when Merb was still a thing. It seems
to
still work, but you might still want to read it to understand what it
does.
It’s not that long:

What I was trying to ask is whether there couldn’t be
a Kernel#autoload_relative call or a ‘relative: true’
optional parameter to Kernel#autoload to make it symmetric
with the require/require_relative combination…

There is one thing I would much rather have than this: Enough hooks
into
either the language or the autoload mechanism so that things like this
are
possible to do entirely inside Ruby.

That is: Currently, we have const_missing, but it isn’t called when
someone
tries to define a constant. For example, if I have Rails-style
const_missing
hacks, and I want to re-open a previously-defined module, I need to do
this:

Foo::Bar.module_eval do
…
end

instead of:

module Foo
module Bar
…
end
end

The latter will never hit const_missing, and would thus never load
foo.rb or
whatever. And sure, if I’m re-opening Bar, I’m probably better off
defining a
separate module and forcing Bar to include it. But what if this is
inside
foo/bar.rb? Now if I say

require ‘foo/bar’

I’ll get Foo::Bar, but I won’t ever load foo.rb – whereas, say,

Foo.module_eval do
module Bar
…
end
end

…ew.

Anyway, what I really want is something like:

autoload :Foo do
# block to ensure Foo is defined
end

In this case, autoload is the wrong word, but this would let me write
the
ultimate autoload-based replacement for the old Rails const_missing
hack. What
I have now is in the ‘autoloader’ gem:

AutoLoader << ‘/path/to/lib’

It will crawl ‘lib’ one level deep, take every ‘.rb’ file, and create an
‘autoload’ tag for it. But in order to map Foo::Bar to ‘foo/bar.rb’, I
still
need the user to have a foo.rb file which does some extra work:

module Foo
include AutoLoader
end

I can’t think of any way to make this work other than having some
callback
fire when the file actually gets loaded. Last I checked, Kernel#autoload
would
call ‘require’ from C when fired, so overloading ‘require’ doesn’t help.
And
const_missing isn’t a substitute, nor is there any way to duplicate
Kernel#autoload’s functionality from Ruby.

I actually think autoloading via require_relative is a bad idea, versus
just
adding something to the load path. Some people think autoload itself is
a bad
idea. There are probably workarounds enough for both of us. But it does
frustrate me that Kernel#autoload is so inflexible, and it really
doesn’t seem
like it needs to be.

10

On Fri, Jun 17, 2011 at 9:33 AM, David M. [email protected]
wrote:

If everything in your ‘lib’ hierarchy is normally loaded via autoload, and
Ok, there’s some FILE ugliness, but in exchange, you automagically get
‘autoload’ statements generated for everything, and you don’t have to modify
your load path.

I think there is a cure for that:

module AutoLoader
def self.included(cl)
# “alc.rb:33:in `class:X’”
file = caller[1][%r{\A(.*?):\d+:in\s}, 1]
self << File.dirname(file)
end

debug:

def self.<<(d)
puts “Adding directory: #{d}”
end
end

AutoLoader << ‘/path/to/lib’

It will crawl ‘lib’ one level deep, take every ‘.rb’ file, and create an
‘autoload’ tag for it. But in order to map Foo::Bar to ‘foo/bar.rb’, I still
need the user to have a foo.rb file which does some extra work:

module Foo
include AutoLoader
end

In Ruby land we could at least do

auto_module :Foo do

no more “include” needed here

end

or even

auto_module :Foo

if there is nothing that needs to be defined here. But then we have a
“require” statement and a file with a single line - that doesn’t feel
right.

like it needs to be.
Hm, for me this generally works pretty well: I have a lib directory
which is part of the load path and for larger libraries I have an
initial file (e.g. “foo.rb”) where the root namespace is defined
together with a set of autoload directives which then load files like
“foo/class1”, “foo/class2” etc.

Basically, as long as there is no fixed relationship between file name
and content, there will always be a level of manual intervention
needed. True, you can have conventions manifest in code (like your
AutoLoader) but as long as there is no predominant convention that
fits all needs we won’t have a generalized solution. The advantage of
the current autoload approach is that it makes things explicit.
Downside is still that the relativity issue (with regard to pathnames,
not E=mc2) is still there.

But that can be solved with

class Module
def autoload_r(const, path)
file = caller[1][%r{\A(.*?):\d+:in\s}, 1]
autoload(const, File.join(File.dirname(file), path))
end
end

Kind regards

robert

11

On Friday, June 17, 2011 06:06:33 AM Robert K. wrote:

to modify your load path.

debug:

def self.<<(d)
puts “Adding directory: #{d}”
end
end

That needs some work. Dirname isn’t enough, since right now, things may
be
nested arbitrarily deep – I could have lib/foo/bar.rb which looks
similar:

module Foo
module Bar
include AutoLoader
end
end

But I clearly still want ‘lib’ to be the top-level directory for this
magic.
That is, I want foo/bar/baz.rb to map to Foo::Bar::Baz, not to Bar::Baz.

I really don’t mind the FILE ugliness, given that it only has to
happen
once, at the very top of the project. The assumption is that you’d never
have
a reason to require anything other than this top-level library, since
everything else is included with autoload, so you’ll only get the stuff
you
need – that is, there’s no advantage to:

require ‘foo/bar’
Foo::Bar.do_something!

rather than

require ‘foo’
Foo::Bar.do_something!

So I have no motivation to get rid of FILE, though maybe there
should be
some convenience version, maybe something like:

AutoLoader.add_relative_path ‘path/to/lib’ # defaults to ‘.’

if there is nothing that needs to be defined here. But then we have a
“require” statement and a file with a single line - that doesn’t feel
right.

Yeah, it doesn’t really solve the problem. ‘include’ isn’t the problem.
The
problem is that this library shouldn’t force me to add any special
syntax or
calls, let alone special files, to make it work. It should just let me
pretend
that my entire directory got slurped – this was inspired by Rails’
const_missing behavior, but also by Ramaze’s “acquire” which just
requires
every ruby file in a given path.

Hm, for me this generally works pretty well: I have a lib directory
which is part of the load path and for larger libraries I have an
initial file (e.g. “foo.rb”) where the root namespace is defined
together with a set of autoload directives which then load files like
“foo/class1”, “foo/class2” etc.

I don’t mind this, except for two things:

  • I shouldn’t have to modify the load path. I should be doing it
    anyway,
    maybe, but it shouldn’t be a requirement.

  • It’s annoying having to explicitly autoload everything, especially
    when I’m
    usually following a convention.

Basically, as long as there is no fixed relationship between file name
and content, there will always be a level of manual intervention
needed.

I like convention over configuration. Not convention instead of
configuration
– there’s always manual autoload if I need them, and I suspect the
first
thing I’d change about AutoLoader at this point is being able to specify
which
files are included and which aren’t. But when 99% of the time, the
process is:

  • Create foo/bar.rb.
    • Create class Foo::Bar in foo/bar.rb.
    • Add autoload line to foo.rb.
  • Did I really want it called bar.rb? Maybe baz.rb is a better name.
    • git mv foo/bar.rb foo/baz.rb
    • Change Foo::Bar to Foo::Baz in foo/baz.rb
    • Change autoload line in foo.rb.
    • Update any other references.
  • Maybe I want to split it into foo/one.rb and foo/two.rb.
    • create foo/one.rb, foo/two.rb.
    • edit/move some stuff, update references.
    • git add foo/one.rb foo/two.rb
    • git rm foo/baz.rb
    • edit foo.rb and update the autoload line.

This is annoying. Sure, it’s going to be annoying no matter what, but
anything
I can do to ease this pain is helpful, because I do this kind of thing a
lot
in the first few hours of a project. Anytime I see a better
organization,
now is the time, before I’ve published it to a dev team (let alone to
the
world via Github) and locked myself into an API.

It’s also annoyingly redundant and menial. This is something the machine
can
do perfectly well for me, so why not let it?

Also worth mentioning: When you’re autoloading everything, anything
beyond the
top-level “require ‘yourgem’” is no longer part of your public API. So,
any
naming convention is entirely for your own convenience at this point. I
just
did what Rails did and made it convenient to follow that one naming
convention. So when I actually use AutoLoader, I don’t have exceptions
to that
naming convention, or, really, any reason to.

The advantage of
the current autoload approach is that it makes things explicit.
Downside is still that the relativity issue (with regard to pathnames,
not E=mc2) is still there.

Well, as you demonstrate, that’s easily solved. But in my opinion, the
other
downside is that it makes things explicit, when there isn’t a good
reason for
them to be.

At this point, AutoLoader isn’t about solving the relativity issue, it’s
about
DRYing things up a bit.