# Memory Consumption issues **URL:** \ **Category:** Rails\ **Created:** [April 1, 2008, 8:49pm UTC](https://www.ruby-forum.com/t/memory-consumption-issues/133575 "2008-04-01T20:49:14Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![djohn](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/cab0a1/32.png) [@djohn](https://www.ruby-forum.com/u/djohn)\ **Post date:** [April 1, 2008, 8:49pm UTC](https://www.ruby-forum.com/t/memory-consumption-issues/133575/1 "2008-04-01T20:49:14Z")
Greetings, I’ve recently deployed an app that is having some serious memory consumption issue. Dropping mongrels quite frequently when the threshold is met. Initially, we chalked this problem up to RMagick (I know, I know, but we needed watermarking). We quickly optimized things so all images were only processed once then neatly stored in their own folder. When an image is requested, we confirm that it doesn’t exist before processing it. Been tested, and this is working as it should. According to some memory profiling we’ve done, there are a couple of key pages that are creating large amounts of strings that are not being reclaimed. Has anyone ever experienced issues similar to this? Cheers, DJ ---
**Author:** ![djohn](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/cab0a1/32.png) [@djohn](https://www.ruby-forum.com/u/djohn)\ **Post date:** [April 1, 2008, 10:31pm UTC](https://www.ruby-forum.com/t/memory-consumption-issues/133575/2 "2008-04-01T22:31:02Z")
What is memory limit per process? ---
**Author:** ![djohn](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/cab0a1/32.png) [@djohn](https://www.ruby-forum.com/u/djohn)\ **Post date:** [April 1, 2008, 10:52pm UTC](https://www.ruby-forum.com/t/memory-consumption-issues/133575/3 "2008-04-01T22:52:29Z")
Dave - On 1-Apr-08, at 2:49 PM, Dave J. wrote: > only processed once then neatly stored in their own folder. When an > Cheers, > DJ I too am presently using monit to manage this until I can track down the leak. Our mongrel’s high water mark is 280 mb, and each of the 11 restart a few times per day. Rmagick is in place, but isolated, and likely not the source. There’s an article somewhere (lost in my booksmarks) that helps track down leaks. As I suspect my leaks are very specific, I have not appealed to the general group, but perhaps this thread will turn out something interesting. Jodi ---
**Author:** ![djohn](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/cab0a1/32.png) [@djohn](https://www.ruby-forum.com/u/djohn)\ **Post date:** [April 2, 2008, 12:31am UTC](https://www.ruby-forum.com/t/memory-consumption-issues/133575/4 "2008-04-02T00:31:26Z")
Thanx Dave - On 1-Apr-08, at 5:10 PM, Dave J. wrote: > [Memory leak profiling with Rails :: scottstuff.net](http://scottstuff.net/blog/articles/2006/08/17/memory-leak-profiling-with-rails) > > We’ve also been using bleak\_house: > [http://blog.evanweaver.com/files/doc/fauna/bleak\_house/files/README.html](http://blog.evanweaver.com/files/doc/fauna/bleak_house/files/README.html) > yet still can’t nail down exactly whats going on here. > > Jodi, what version of Rails are you using for you project? > > Thanks, > Dave We’re on 1.2.3, mysql 5, ubunutu 6.06 LTS, ruby 1.8.6 (i686). I’m deploying a new server (throw money at the problem), but will be getting to the leaks next week. We’ve had the leaks since day 1, and although we have 1.8million plus pages, controller actions are limited - so I should be able to localize the area quickly. I’ll update this thread with anything learned. thank you again Dave for the references. Jodi ---
**Author:** ![djohn](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/cab0a1/32.png) [@djohn](https://www.ruby-forum.com/u/djohn)\ **Post date:** [April 2, 2008, 2:14am UTC](https://www.ruby-forum.com/t/memory-consumption-issues/133575/5 "2008-04-02T02:14:07Z")
On Apr 1, 2:49 pm, Dave J. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > Greetings, > > I’ve recently deployed an app that is having some serious memory > consumption issue. Dropping mongrels quite frequently when the > threshold is met. > > Initially, we chalked this problem up to RMagick (I know, I know, but we > needed watermarking). If RMagick is not the source of leaks currently then this won’t help, but I just do the following: `convert -resize 480x360 #{temp_file_path} #{file_path}` My image uploading is low volume, so this works fine for me. I think it would work well even for much higher volumes, but I haven’t benchmarked the difference between a command line invocation and RMagick. It’s not likely to leak though. If you track down the problem please post it. ---
**Author:** ![djohn](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/cab0a1/32.png) [@djohn](https://www.ruby-forum.com/u/djohn)\ **Post date:** [April 1, 2008, 11:10pm UTC](https://www.ruby-forum.com/t/memory-consumption-issues/133575/6 "2008-04-01T23:10:46Z")
Hey Jodi, I believe our mongrels memory threshold is in the same ballpark. Here are a couple of articles we’ve been looking at, following along with these processes and using the memory analysis tool in the second has revealed that the strings not getting reclaimed are blank… [http://codehappy.wordpress.com/2007/01/17/tracking-a-memory-leak-in-rails/](http://codehappy.wordpress.com/2007/01/17/tracking-a-memory-leak-in-rails/) > **[Memory leak profiling with Rails :: scottstuff.net](https://scottstuff.net/blog/articles/2006/08/17/memory-leak-profiling-with-rails/index.html)** > > One of my long-running problems with Rails (and Ruby in general) is that it’s difficult to debug memory leaks. I’ve had a number of cases where I’ve stuck something into a long-lived array or hash and discovered much later that my Ruby process was... We’ve also been using bleak\_house: [http://blog.evanweaver.com/files/doc/fauna/bleak\_house/files/README.html](http://blog.evanweaver.com/files/doc/fauna/bleak_house/files/README.html) yet still can’t nail down exactly whats going on here. Jodi, what version of Rails are you using for you project? Thanks, Dave ---
**Author:** ![djohn](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/cab0a1/32.png) [@djohn](https://www.ruby-forum.com/u/djohn)\ **Post date:** [April 7, 2008, 6:03pm UTC](https://www.ruby-forum.com/t/memory-consumption-issues/133575/7 "2008-04-07T18:03:07Z")
Greetings mem-leakers On 1-Apr-08, at 8:13 PM, Brian A. wrote: > > needed watermarking). > > If you track down the problem please post it. I don’t have any answers yet, a few clues, but have some tools to suggest: re: clues. this article looks promising: [http://blog.pluron.com/2008/01/ruby-on-rails-i.html](http://blog.pluron.com/2008/01/ruby-on-rails-i.html) As for tools, I’m trying out Evan W.'s bleak-house: [http://blog.evanweaver.com/files/doc/fauna/bleak\_house/files/README.html](http://blog.evanweaver.com/files/doc/fauna/bleak_house/files/README.html) as it patches ruby, you’ll want to use this on your staging server (Evan says he uses the patched ruby in production) this is the output from Bleak: ./homestars/shared/log $ bleak /tmp/bleak.3242.0.dump 632837 total objects Final heap size 632837 filled, 732169 free Displaying top 20 most common line/class pairs 510101 **null** : **null** : **node** 29450 **null** : **null** :String 5511 /usr/local/lib/ruby/gems/1.8/gems/memcached-0.8.1/lib/ memcached/memcached.rb:295:String 4692 (eval):1: **node** 3442 /var/www/apps/homestars/releases/20080407140541/vendor/rails/ actionpack/lib/action\_controller/routing.rb:836:String 3442 /var/www/apps/homestars/releases/20080407140541/vendor/rails/ actionpack/lib/action\_controller/routing.rb: 836:ActionController::Routing::DividerSegment 2084 **null** : **null** :Regexp 1902 /usr/local/lib/ruby/gems/1.8/specifications/ tzinfo-0.3.7.gemspec:12:String 1686 generated\_code\_(/var/www/apps/homestars/releases/ 20080407140541/vendor/rails/actionpack/lib/action\_controller/ routing.rb:388):1: **node** 1491 /var/www/apps/homestars/releases/20080407140541/vendor/rails/ actionpack/lib/action\_controller/routing.rb:834:String 1491 /var/www/apps/homestars/releases/20080407140541/vendor/rails/ actionpack/lib/action\_controller/routing.rb: 834:ActionController::Routing::StaticSegment 1287 /var/www/apps/homestars/releases/20080407140541/vendor/rails/ activerecord/lib/…/…/activesupport/lib/active\_support/deprecation.rb: 92: **node** 1106 /var/www/apps/homestars/releases/20080407140541/vendor/rails/ actionpack/lib/action\_controller/routing.rb: 827:ActionController::Routing::DynamicSegment 1056 /usr/local/lib/ruby/gems/1.8/specifications/ rmagick-2.2.2.gemspec:13:String 1051 /usr/local/lib/ruby/1.8/yaml.rb:133:String 976 /var/www/apps/homestars/releases/20080407140541/vendor/rails/ activerecord/lib/…/…/activesupport/lib/active\_support/core\_ext/ module/aliasing.rb:28: **node** 938 /var/www/apps/homestars/releases/20080407140541/vendor/rails/ activerecord/lib/…/…/activesupport/lib/active\_support/core\_ext/ module/aliasing.rb:29: **node** 891 **null** : **null** :Class 845 /var/www/apps/homestars/releases/20080407140541/vendor/rails/ actionpack/lib/action\_controller/routing.rb: 945:ActionController::Routing::Route 845 /var/www/apps/homestars/releases/20080407140541/vendor/rails/ actionpack/lib/action\_controller/routing.rb:865:Hash (the above is from a single run of hitting a single url for 15 minutes. working out a test approach presently) The above is after a 30 minute run - showing 5511 objects lost in memached (funny enough also written by Evan - I’ve asked him about the leak). The \_\_ leaks are supposedly internal to ruby - boy, that’s a load of lost objects - other than tzinfo (which I’ve heard is a culprit), it seems to me that routing is generating a load. Anybody have any info on routing and leaks? I’m at a bit of a loss when digging info ruby leaks/garbage collection. If anybody has a good ref I’d appreciate it. rails 1.2.6 ruby 1.8.6 (2008-03-03 patchlevel 901) [i686-linux] thanx. Jodi ---
**Author:** ![djohn](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/cab0a1/32.png) [@djohn](https://www.ruby-forum.com/u/djohn)\ **Post date:** [April 7, 2008, 10:04pm UTC](https://www.ruby-forum.com/t/memory-consumption-issues/133575/8 "2008-04-07T22:04:40Z")
Hey all, We’ll we tracked down our memory issues. We had some complex join queries going on and it appears that the AR objects created from said queries were not getting reclaimed. So if you are still having issues, and have some complex find\_by\_sql or find statements using a lot/nested includes. Try breaking those up into two queries… Vast improvement here after doing that. Cheers, Dave ---
**Author:** ![djohn](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/cab0a1/32.png) [@djohn](https://www.ruby-forum.com/u/djohn)\ **Post date:** [April 7, 2008, 10:10pm UTC](https://www.ruby-forum.com/t/memory-consumption-issues/133575/9 "2008-04-07T22:10:07Z")
Thanx Dave - On 7-Apr-08, at 4:04 PM, Dave J. wrote: > into > two queries… Vast improvement here after doing that. > > Cheers, > Dave I’m interested in how your determined they weren’t be reclaimed? ie. how did you zero in on the opportunity? Jodi