Forum: Ruby require_all 1.0.0: A wonderfully simple way to load your code

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
2f55791ab9018b4d01fb741fab21843d?d=identicon&s=25 Tony Arcieri (Guest)
on 2009-06-09 11:22
(Received via mailing list)
require_all

A wonderfully simple way to load your code.

http://github.com/tarcieri/require_all/tree/master

Tired of futzing around with require statements everywhere, littering
your
code with require File.dirname(__FILE__) crap? What if you could just
point
something at a big directory full of code and have everything just
automagically load regardless of the dependency structure?

Wouldn’t that be nice? Well, now you can!
require 'require_all'

You can now require_all in a multitude of different ways:
require_all *args

One of the easiest ways to require_all is to give it a glob, which will
enumerate all the matching files and load them in the proper order. For
example, to load all the Ruby files under the ‘lib’ directory, just do:
require_all 'lib/**/*.rb'

If the dependencies between the matched files are unresolvable, it will
throw the first unresolvable NameError.

Don’t want to give it a glob? Just give it a list of files:
require_all Dir.glob("blah/**/*.rb").reject { |f| stupid_file(f) }

Or if you want, just list the files directly as arguments:
require_all 'lib/a.rb', 'lib/b.rb', 'lib/c.rb', 'lib/d.rb'

It’s just that easy! Code loading shouldn’t be hard.
Methodology

I didn’t invent the approach this gem uses. It was shamelessly stolen
from
Merb. Once upon a time at MountainWest RubyConf we were discussing how
horrible ActiveSupport’s dependencies.rb hijacking of const_missing and
someone described the approach Merb used to me. It was so simple and
clean!
Here’s how it works:

   1. Enumerate the files in the glob
   2. Try to load all of the files. If we encounter a NameError loading
a
   particular file, store that file in a “try to load it later” list.
   3. If all the files loaded, great, we’re done! If not, go through the
   “try to load it later” list again rescuing NameErrors the same way.
   4. If we walk the whole “try to load it later” list and it doesn’t
shrink
   at all, we’ve encountered an unresolvable dependency. In this case,
   require_all will rethrow the first NameError it encountered.

Questions? Comments? Concerns?

You can reach the author on github or freenode: “tarcieri”

Or by email: tony@medioh.com

Got issues with require_all to report? Post ‘em here:

Github Tracker <http://github.com/tarcieri/require_all/issues>
License

MIT (see the LICENSE file for details)
45196398e9685000d195ec626d477f0e?d=identicon&s=25 Thomas Sawyer (7rans)
on 2009-06-09 15:06
(Received via mailing list)
On Jun 9, 5:21 am, Tony Arcieri <t...@medioh.com> wrote:
>
>
> Methodology
>    3. If all the files loaded, great, we’re done! If not, go through the
>
> Got issues with require_all to report? Post ‘em here:
>
> Github Tracker <http://github.com/tarcieri/require_all/issues>
> License

  require 'facets/kernel/require_all'

The idea predates Merb. However, it's always good too see how
implementations may have improved over time.

Also, I have seen another definition of #require_all floating around
that simply loads every .rb file in a given directory. I wonder if
both conceptions of this function (glob loading and directory loading)
could be integrated into one. Eg.

  require_all 'some/path'

would do the same as

  require_all 'some/path/*.rb'

Cheers,
T.
Aee77dba395ece0a04c688b05b07cd63?d=identicon&s=25 Daniel Berger (djberg96)
on 2009-06-09 16:37
(Received via mailing list)
> A wonderfully simple way to load your code.
> Wouldn't that be nice? Well, now you can!
> If the dependencies between the matched files are unresolvable, it will
>
> I didn't invent the approach this gem uses. It was shamelessly stolen
> from
> Merb

Trust me, Merb was not the first library to come up with Java
style/globbing
import statements. I remember this being discussed a *long* time ago,
and it
generally crops up from time to time on the mailing list. I'm pretty
sure it
was an RCR at one point (for altering the behavior of Ruby's own
require),
but I can't find it now.

But, now that I'm looking on the RAA, I can't find a library for it. So,
still a nice thing to have, thanks. :)

Regards,

Dan
2f55791ab9018b4d01fb741fab21843d?d=identicon&s=25 Tony Arcieri (Guest)
on 2009-06-09 19:31
(Received via mailing list)
On Tue, Jun 9, 2009 at 7:06 AM, trans <transfire@gmail.com> wrote:

>
>   require 'facets/kernel/require_all'
>

Bah, should've checked facets first, but that really is the problem with
facets... it's a big kitchen sink and you never know what you're going
to
find in there or not

That said, this was yanked out of a larger system and incorporates some
hacks for living along ActiveSupport and the boogiemen which lurk inside
dependencies.rb.

The idea predates Merb. However, it's always good too see how
> implementations may have improved over time.
>

Thanks!  I figure if nothing else it could have a more powerful API.


>
That seems as simple as checking if the given path is a directory.
Perhaps
it'd be a nice feature for 1.0.1
2f55791ab9018b4d01fb741fab21843d?d=identicon&s=25 Tony Arcieri (Guest)
on 2009-06-09 21:00
(Received via mailing list)
On Tue, Jun 9, 2009 at 7:06 AM, trans <transfire@gmail.com> wrote:

>
You ask for it, you get it!

I've just released require_all 1.0.1, which in addition to all the other
functionality accepts a directory name as an argument and will glob all
the
*.rb files underneath it.

Figured I'd spare the list an additional release announcement :)
Bec38d63650c8912b6ba9b557fb953b9?d=identicon&s=25 Roger Pack (rogerdpack)
on 2009-06-10 19:52
> Tired of futzing around with require statements everywhere, littering
> your
> code with require File.dirname(__FILE__) crap? What if you could just
> point
> something at a big directory full of code and have everything just
> automagically load regardless of the dependency structure?

Wow I really like this idea.
47b1910084592eb77a032bc7d8d1a84e?d=identicon&s=25 Joel VanderWerf (Guest)
on 2009-06-10 20:12
(Received via mailing list)
Roger Pack wrote:
>> Tired of futzing around with require statements everywhere, littering
>> your
>> code with require File.dirname(__FILE__) crap? What if you could just
>> point
>> something at a big directory full of code and have everything just
>> automagically load regardless of the dependency structure?
>
> Wow I really like this idea.

Hm, what about

module Foo
   class HonkinBigResource
   end

   RESOURCE_LIST << HonkinBigResource.new
     # defer until RESOURCE_LIST defined, if necessary

   class Bar < Base
     # Base may be undefined, so defer loading of this file
   end
end

This could result in wasted time recreating a resource, duplicates on
the list, etc.

Sure, it's possible to program defensively to avoid the above, but I'd
rather set up a proper dependency structure with explicit requires.

Also, all uses of defined?(SomeClass) will have more or less random
results, won't they?

OTOH, maybe this is like static vs. dynamic typing, and I'm just too old
school to get it ;)
2f55791ab9018b4d01fb741fab21843d?d=identicon&s=25 Tony Arcieri (Guest)
on 2009-06-11 20:34
(Received via mailing list)
On Wed, Jun 10, 2009 at 12:11 PM, Joel VanderWerf
<vjoel@path.berkeley.edu>wrote:

>    # Base may be undefined, so defer loading of this file
>  end
> end
>
> This could result in wasted time recreating a resource, duplicates on the
> list, etc.
>

Yes, there are potential issues with this approach if you do that sort
of
thing.

The larger issues are when you use this with something like
ActiveSupport
loaded which redefines const_missing on everything.  This will invoke
that
const_missing callback whenever it hits a missing constant, and in
ActiveSupport its const_missing handler is an absolute nightmare.

Also, all uses of defined?(SomeClass) will have more or less random
results,
> won't they?
>

If you have code in the toplevel or a class/method body which is using
"defined?" then it is possible you will get nondeterministic results
depending on the order code is loaded in (i.e. there may be multiple
possible orderings which satisfy all dependencies), provided those
constants
are getting defined in another file which you're pulling in through
require_all.

However, if you have a file which defines a bunch of constants (e.g. a
config file) you can just load that file first, then use require_all to
pull
in the rest of your code.

The behavior of "defined?" inside of methods will work exactly as it
always
did since that code isn't actually executed until the method is invoked.
47b1910084592eb77a032bc7d8d1a84e?d=identicon&s=25 Joel VanderWerf (Guest)
on 2009-06-11 21:00
(Received via mailing list)
Tony Arcieri wrote:
> The behavior of "defined?" inside of methods will work exactly as it always
> did since that code isn't actually executed until the method is invoked.

True, but sometimes methods get invoked at load time...
Bec38d63650c8912b6ba9b557fb953b9?d=identicon&s=25 Roger Pack (rogerdpack)
on 2009-07-15 17:58
> Trust me, Merb was not the first library to come up with Java
> style/globbing
> import statements. I remember this being discussed a *long* time ago,
> and it
> generally crops up from time to time on the mailing list. I'm pretty
> sure it
> was an RCR at one point (for altering the behavior of Ruby's own
> require),
> but I can't find it now.

Hmm.  I too wish that ruby core had something like

require_relative 'lib/a.rb'
or require_all 'lib/*.rb'
or require_all 'lib' => require_all 'lib/*.rb' or require_all
'lib/init.rb'

Thoughts?
=r
2f55791ab9018b4d01fb741fab21843d?d=identicon&s=25 Tony Arcieri (Guest)
on 2009-07-15 21:56
(Received via mailing list)
On Wed, Jul 15, 2009 at 9:58 AM, Roger Pack <rogerpack2005@gmail.com>
wrote:

> Hmm.  I too wish that ruby core had something like
>
> require_relative 'lib/a.rb'
> or require_all 'lib/*.rb'
> or require_all 'lib' => require_all 'lib/*.rb' or require_all
> 'lib/init.rb'
>
> Thoughts?
>

I really would like to see this in core for one reason: with a pure Ruby
solution it's incredibly difficult if not impossible to play nice with
ActiveSupport's dependencies.rb.  The require_all gem contains some
hacks
specific to ActiveSupport to try to play nicely, but I've still run into
cases where I have to adjust code to keep various mechanisms in
ActiveSupport's dependency loader from freaking out.

I am really curious how rails core plans to get the Merb and
ActiveSupport
approaches to loading code working nicely together, or for that matter,
why
Merb users using ActiveRecord as their ORM (and vicariously
ActiveSupport)
don't run into "is not missing constant" errors more frequently,
especially
in multithreaded programs.
Bec38d63650c8912b6ba9b557fb953b9?d=identicon&s=25 Roger Pack (rogerdpack)
on 2009-07-15 22:09
Tony Arcieri wrote:
> On Wed, Jul 15, 2009 at 9:58 AM, Roger Pack <rogerpack2005@gmail.com>
> wrote:
>
>> Hmm.  I too wish that ruby core had something like
>>
>> require_relative 'lib/a.rb'
>> or require_all 'lib/*.rb'
>> or require_all 'lib' => require_all 'lib/*.rb' or require_all
>> 'lib/init.rb'
>>
>> Thoughts?
>>
>
> I really would like to see this in core for one reason: with a pure Ruby
> solution it's incredibly difficult if not impossible to play nice with
> ActiveSupport's dependencies.rb.

Yeah I wish active support would just go away.  It seems unstable, even
today.

So my thought for core would be to suggest either require_relative
'lib/a.rb' or perhaps require 'directory_name' which (like Python) loads
directory_name/init.rb or what not.

My preference would be advocate require_relative, having never seemed to
miss the require 'directory_name'

Thoughts?
=r
This topic is locked and can not be replied to.