Minimizing memory allocations

Ruby
21

On Jan 27, 2006, at 11:04 PM, Alex C. wrote:

module Memoize
MEMOIZE_VERSION = “1.2.0”

Set a constant to the module version, so user code can check it as
needed.

Memoize the method +name+. If +file+ is provided, then the

method results

are stored on disk rather than in memory. This consumes

virtually no

memory and is persistant.

def memoize(name, file=nil)
meth = method(name)

Grab a reference to the original method.

  if file
     cache = Hash.new.update(Marshal.load(File.read(file)))

rescue {}
else
cache = {}
end

Load an existing cache file, if requested and one exists. Otherwise,
set the cache to an empty Hash.

  if file
     (class << self; self; end).class_eval do
        define_method(name) do |*args|
           unless cache.has_key?(args)
              cache[args] = meth.call(*args)
              File.open(file, "wb+"){|f| Marshal.dump(cache, f) }
           end
           cache[args]
        end
     end

This is the singleton class trick James B. described. It this
version, the cache is checked for an entry with all the arguments it
was just called with. If it’s not found, the original method is
triggered to add that entry to the cache. The file is also modified
to contain the new entry. Either way, the cache now has the needed
entry, which is returned.

  end

Exact same thing, minus the file dump. In fact, that whole if/else/
end chunk of code could be simplified to:

      (class << self; self; end).class_eval do
         define_method(name) do |*args|
            unless cache.has_key?(args)
               cache[args] = meth.call(*args)
               File.open(file, "wb+"){|f| Marshal.dump(cache,

f) } if file
end
cache[args]
end
end

  cache

Return the cache we will use for user code to examine/modify, if
desired.

end

Hope that helps.

James Edward G. II

22

James Edward G. II wrote:

On Jan 28, 2006, at 12:39 AM, James B. wrote:

Basically, the arguments to the target method are used as a hash key
into a cache. The cache hash is stored either in memory, or
(optionally) on disk.

It’s actually always in memory with the current implementation, and can
optionally be duplicated on disk.

Ah, I missed that. Thanks.

Really nice lib, and the use of singleton methods suggests other ideas.

Thanks,

James B.

23

Alex C. wrote:

…

Yikes! This is definitly something I’m going to need to learn to use, and use
as often as possible. So anywho, took a quick peek at memoize.rb and was
again just blown away but this time because the entire file was only 39 lines
of code including white space! How can this be?

Unfortuantely, there are no comments to be had, so I was wondering
if someone would be so kind as to reply with inserted comments in the code,
(yes I know ruby is supposed to be readable, but im new, ok? :slight_smile:
Just as a precaution so that I don’t make any weird assumptions,
something I’m prone to do. This would be greatly appreciated.

Basically, the arguments to the target method are used as a hash key
into a cache. The cache hash is stored either in memory, or (optionally)
on disk.

If you give memoize a file name it will serialize the results cache to
and from disk using Ruby’s Marshal code.

Otherwise, the cache just sits in memory.

The code defines a singleton method of the same name as the target
method. Calls to the target method are passed to this singleton method;
it has first crack at looking to see if there is a cached value.

If it has the value, then it is returned; all done.

Otherwise, the value is obtained by calling the original method defined
in the class source code, the results are cached (and the file updated,
if defined) and the value returned.

The neat part is the use of a singleton method to intercept calls to the
target method:

class Foo

include Memoize

def initialize
memoize :baz
end

def baz val
“Baz sez #{val}”
end

end

When you call

foo = Foo.new
foo.baz( ‘Hey.’ )

the first baz invoked is the singleton method defined on foo. If it
does not have a cached value, it calls the baz method defined in the Foo
class.

puts foo.singleton_methods # baz

(But note that an in-memory hash will exist for each instance. A file
allows for all instances to share values, and to have those values
persist from runs of the application, but may introduce race
conditions.)

–
James B.

“Blanket statements are over-rated”