Where does all the memory go?

Rails
1

I’ve a method within an Event model…

def batch_import(folder)
require ‘find’

  Find::find("#{SOURCE_PATH}/#{folder}") do |path|
      if path[-3..-1].downcase == 'jpg'
        @photo = Picture.new()
        @photo.data = File.new(path)
        @photo.filename = File.basename(path)
        @photo.caption = "none"
        self.pictures << @photo
        self.save
      end
  end

end

Where Picture is …

class Picture < FlexImage::Model
file_store ‘pictures’

belongs_to :event, :counter_cache => true
acts_as_list :scope => :event

acts_as_solr :fields => [:caption]

end
and Event is…

class Event < ActiveRecord::Base
has_many :pictures, :order => ‘position’,:dependent=>:destroy

When I run the code it uses a massive 1.5GB of memory to process 28
pictures. Ruby eventually returns the memory once you’ve browsed to
another view, but still?!?

Anyone have any ideas how/why?
should I be forcing garbage collection on @photo after each image?!?

2

On 8/13/07, kevin evans [email protected] wrote:

        @photo.filename = File.basename(path)

file_store ‘pictures’
has_many :pictures, :order => ‘position’,:dependent=>:destroy

When I run the code it uses a massive 1.5GB of memory to process 28
pictures. Ruby eventually returns the memory once you’ve browsed to
another view, but still?!?

Anyone have any ideas how/why?
should I be forcing garbage collection on @photo after each image?!?

I don’t know if that’ll matter. It looks like FlexImage is reading the
files into memory, and it still might be referenced in Event’s
self.pictures array.

–
Rick O.
http://lighthouseapp.com
http://weblog.techno-weenie.net
http://mephistoblog.com

3

Have you tried just doing the Find::find stuff without a body and seeing
if that accumulates a ton of memory? If it doesn’t, then you might want
to explicitly nil out the @photo after it’s been saved. Likewise, it
look like you never close all the Files you are opening. You might want
to watch out for that.

Finally, how big are these jpgs? Are they super-compressed, so that they
actually do take up a ton of memory when loaded?

4

Thanks for the replies.

Bryan, Soory I’m a bit of a ruby newbie (still!!!), do you mean use
some form of syntax for the Find that returns all images as an array,
for processing?

also @photo = nil (is that what you mean?

On Aug 13, 10:17 pm, Bryan D. [email protected]

5

Actually, on further testing it does look like it the 'self.pictures
<< @photo ’ that eats up all the memory. Yes, each image is very
large. So time to look for a new solution. Shame.

6

kevin evans wrote:

Thanks for the replies.

Bryan, Soory I’m a bit of a ruby newbie (still!!!), do you mean use
some form of syntax for the Find that returns all images as an array,
for processing?

also @photo = nil (is that what you mean?

On Aug 13, 10:17 pm, Bryan D. [email protected]

In my first suggestion, I’m saying that you should try

  Find::find("#{SOURCE_PATH}/#{folder}") do |path|
  end

and watch the memory usage of just that alone and see if it’s poor. I’m
not familiar with that library personally, so I’d want to verify that
it’s not the memory hog.

You have it right on the other one, @photo = nil will make sure that
reference goes away which could lead to quicker garbage collection.
Actually, it looks like @photo should really be a local variable, since
it’s lifetime should only be within the Find::find block, so drop the @
from @photo.