Forum: Ruby Memory usage with blocks

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
John Ky (Guest)
on 2008-12-02 10:25
(Received via mailing list)
Hi all,

I've got some code, which when run uses a lot of memory (aprox 600 MB
for a
particular given input).  I am trying to figure out why it is using so
much.

When I replace the following code:

                head.play_back do |root|
                   block.call(root)
                end

With this:

                head.play_back(&block)

And make other similar changes.  The program behaves exactly the same
way,
but now uses 200MB less.  Can someone explain to me why this is the
case?

I use blocks everywhere and I'm worried, they're leaking in my code.

joky@linux08:~/wa/bvc-pure-cs/comet/src/swig_client_api/ruby $ ruby -v
ruby 1.8.7 (2008-06-20 patchlevel 22) [i686-linux]

Thanks

-John
John Ky (Guest)
on 2008-12-02 10:39
(Received via mailing list)
Hi,

I think it's the memory profiler I'm using:

           MemoryProfiler.start

Nevermind.

Does anyone know how to properly find memory leaks in a ruby app?

Thanks,

-John
Sebastian H. (Guest)
on 2008-12-02 14:28
(Received via mailing list)
John Ky wrote:
> And make other similar changes.  The program behaves exactly the same way,
> but now uses 200MB less.

Consider the following:

def x(&block)
  block
end

def a(&block)
  tmp = create_a_two_hundred_mega_byte_object()
  x {block.call} # Create a new block which closes over the current
scope
end

def b(&block)
  tmp = create_a_two_hundred_mega_byte_object()
  x(&block) # Reuse the block that has been passed to this method.
            # Do not create a new block
end

If I call b {puts "hello"}, the following will happen:
The block {puts "hello"} is created. This block holds a reference to any
local
variable that has been defined in the scope in which I call method b.
Now method b is called with that block as an argument.
Then a 200mb object is created and stored in tmp.
Now x is called. x returns a Proc representing the block passed to b.
The method b ends, tmp goes out of scope and the 200mb object is garbage
collected.

If I call a {puts "hello"}, this happens:
The block {puts "hello"} is created. This block holds a reference to any
local
variable that has been defined in the scope in which I call method a.
Now method a is called with that block as an argument.
Then a 200mb object is created and stored in tmp.
Now another block is created, which invokes the block passed to a.
This new block holds a reference to any local variable tmp defined in
the
method body of a. Specifically it holds a reference to tmp.
Now x is called. x returns a Proc representing this new block.
The method a ends, but the 200mb is still referenced by the Proc that's
returned from a. So as long as reference to that Proc is stored, the
200mb
object will not be garbage collected.

I don't know whether that's what happens in your code, but it's
certainly
worth looking into.

HTH,
Sebastian
John Ky (Guest)
on 2008-12-03 05:33
(Received via mailing list)
Hi Sebastian,

Why doesn't ruby detect that tmp isn't used in the block and not keep a
reference to it?

I found the offending code:

         def play_back(&block)
            $trace.method_block "PlaybackChain::play_back(&)" do
               sub_block = Proc.new do
                  @tail.play_back do |root|
                     block.call(root)
                  end
               end
               @block.call(sub_block)
            end
         end

         def play_back(&block)
            @heads.each do |head|
               selector.select_each do |selection|
                  block.call(selection)
               end
            end
         end

Replacing those to methods with

         def play_back(&block)
            $trace.method_block "PlaybackChain::play_back(&)" do
               sub_block = Proc.new do
                  @tail.play_back(&block)
               end
               @block.call(sub_block)
            end
         end

         def play_back(&block)
            @heads.each do |head|
               head.play_back(&block)
            end
         end

wiped over 100 MB off the memory usage.

Thanks for your help,

-John

On Tue, Dec 2, 2008 at 11:21 PM, Sebastian H. <
John Ky (Guest)
on 2008-12-03 05:34
(Received via mailing list)
Runs heaps faster too.  Woosh!
Sebastian H. (Guest)
on 2008-12-03 10:20
(Received via mailing list)
John Ky wrote:
> Why doesn't ruby detect that tmp isn't used in the block and not keep a
> reference to it?

Because tmp can still be accessed using the block even if it's not used
in the
block directly:
def foo()
  x = 5
  lambda {puts "Hello world"}
end
hello_world_block = foo
eval("x", hello_world_block) #=> 5

HTH,
Sebastian
This topic is locked and can not be replied to.