Forum: Ruby on Rails Possible solutions to txtdrive process killing for cpu usage

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
982c3f128bb78678033c0bf8c0e226bc?d=identicon&s=25 Saimon Moore (Guest)
on 2006-05-18 12:54
Hi,

I've just deployed a basic ecommerce site on textdrive.com. The
application has been stress tested on our local development machines and
appears to be running fine (no memory leaks).

However, on textdrive the fcgi processes are being killed after just a
few requests apparently because of cpu usage (The log says the processes
exceeds 17% average cpu usage in the last few minutes)

At first it seemed that they we're being killed because the processes
were exceediing the 48Mb resident memory size limit (RSS seemed to
suddenly increase on certain pages (mostly backoffice functions) though
not consistently) but I was pointed to a process_watchdog.log filr which
points to the cpu usage.

Has anyone come across a similar situation? Any hints suggestions on
what I can do about it?

I'd really appreciate any help as although I'm now doing a optimization
refactoring iteration I'm uncertain how I can avoid this.

Regards,

Saimon Moore
921432719b51a266a06878ff4128a7b1?d=identicon&s=25 Dae San Hwang (Guest)
on 2006-05-18 13:02
(Received via mailing list)
I think you should post this to the textdrive forum (http://
forum.textdrive.com/).

On May 18, 2006, at 7:54 PM, Saimon Moore wrote:

> exceeds 17% average cpu usage in the last few minutes)
> what I can do about it?
> Posted via http://www.ruby-forum.com/.
> _______________________________________________
> Rails mailing list
> Rails@lists.rubyonrails.org
> http://lists.rubyonrails.org/mailman/listinfo/rails

Dae San Hwang
daesan@gmail.com
982c3f128bb78678033c0bf8c0e226bc?d=identicon&s=25 Saimon Moore (saimon)
on 2006-05-18 13:17
I already have but what I'm asking for on this list is help or
suggestions with how I could go about tracking/reducing the cpu usage
problem which is rails related.

Dae San Hwang wrote:
> I think you should post this to the textdrive forum (http://
> forum.textdrive.com/).
>
> On May 18, 2006, at 7:54 PM, Saimon Moore wrote:
>
>> exceeds 17% average cpu usage in the last few minutes)
>> what I can do about it?
>> Posted via http://www.ruby-forum.com/.
>> _______________________________________________
>> Rails mailing list
>> Rails@lists.rubyonrails.org
>> http://lists.rubyonrails.org/mailman/listinfo/rails
>
> Dae San Hwang
> daesan@gmail.com
6076c22b65b36f5d75c30bdcfb2fda85?d=identicon&s=25 Ezra Zygmuntowicz (Guest)
on 2006-05-18 18:18
(Received via mailing list)
On May 18, 2006, at 4:17 AM, Saimon Moore wrote:

>>> exceeds 17% average cpu usage in the last few minutes)
>>> what I can do about it?
>>> Posted via http://www.ruby-forum.com/.
>>> _______________________________________________
>>> Rails mailing list
>>> Rails@lists.rubyonrails.org
>>> http://lists.rubyonrails.org/mailman/listinfo/rails
>>
>> Dae San Hwang
>> daesan@gmail.com
>

	Well, what does your app do? What parts of it are processor
intensive? Using RMagick? Some details and we might be able to help you.

-Ezra
982c3f128bb78678033c0bf8c0e226bc?d=identicon&s=25 Saimon Moore (saimon)
on 2006-05-18 18:55
Hi Ezra,

This is what I find strange. Just to choose one example, I have an edit
product view whose controller action simply finds the product and makes
an instance variable out of it so that the form can be populated (i.e.
it's not the product update action).

e.g.

    @product = Product.find(params[:id])
    @song = Song.new
    unless @product && @product.sellable
      @artists = String.new
      @label = Label.new
      @genre = Genre.new
      @instrument = Instrument.new
      @supplier = Supplier.new
    else
      @artists = @product.sellable.artists.collect {|a| a.name }.join('
, ')
      @label = @product.sellable.label.name
      @genre = @product.sellable.genre.name
      @instrument = @product.sellable.instrument.name
      @supplier = @product.supplier.name
    end

The view is just a basic form (created from a few partials) utilizing
standard rails form tags to edit the product. It has a small section
specific to the kind of sellable and a section specific to product
details.

I have suspected Rmagick as a possible offender but after further tests
I have rejected it as a possibility. (Images are processed via rmagick
just the once and thereafter they are cached on the browser)

The Product model:

  has_one :sellable
  belongs_to :supplier #where supplier is sublclassed into Album/Song
(STI)
  belongs_to :image
  has_many :shopping_items, :dependent => :destroy
  has_many :supplier_order_items
  belongs_to :user

There's nothing very complex about this action apart from the fact that
the AR model has a few associations.

I have stress tested this action on my development box (set to
production mode) and it appears rock solid. The fcgi process has a
steady 28Mb during and after a heavy attack (ab -kc 50 -t 300). cpu
levels are similarly non-interesting. valgrind memcheck  doesn't report
anything out of the ordinary

On textdrive though it's a different story, rss starts of at 32Mb and
rockets up after only a few refreshes of this action (and yes I'm
running the app in production mode), the cpu cycles also increase,
though up until I say the process_watchdog.log I wasn't that worried
about them.

In short I'm stumped as to why this is behaving so differently on
textdrive and I'm not really sure what I can do to optimize this.

I'd appreciate any advice.

Thanks,

Saimon
Ezra Zygmuntowicz wrote:
> On May 18, 2006, at 4:17 AM, Saimon Moore wrote:
>
>>>> exceeds 17% average cpu usage in the last few minutes)
>>>> what I can do about it?
>>>> Posted via http://www.ruby-forum.com/.
>>>> _______________________________________________
>>>> Rails mailing list
>>>> Rails@lists.rubyonrails.org
>>>> http://lists.rubyonrails.org/mailman/listinfo/rails
>>>
>>> Dae San Hwang
>>> daesan@gmail.com
>>
>
> 	Well, what does your app do? What parts of it are processor
> intensive? Using RMagick? Some details and we might be able to help you.
>
> -Ezra
6076c22b65b36f5d75c30bdcfb2fda85?d=identicon&s=25 Ezra Zygmuntowicz (Guest)
on 2006-05-18 19:27
(Received via mailing list)
Hi !

On May 18, 2006, at 9:55 AM, Saimon Moore wrote:

>
> a.name }.join('
> details.
> (STI)
> production mode) and it appears rock solid. The fcgi process has a
>
> In short I'm stumped as to why this is behaving so differently on
> textdrive and I'm not really sure what I can do to optimize this.
>
> I'd appreciate any advice.
>
> Thanks,
>
> Saimon



Saimon-

	Yeah that sounds a bit fishy when it works fine on your local box
but not on txtdrv server. What OS are you running locally? I don't
think I have any clear cut answers for you right off the bat here.
Since it seems like a textdrive problem you should probably file a
support ticket there detailing how it works locally but doesn't work
on their servers.

	I no longer use textdrive so I can't say what the issue might be.
Hope you get it figured out though.

Cheers-
-Ezra
982c3f128bb78678033c0bf8c0e226bc?d=identicon&s=25 Saimon Moore (saimon)
on 2006-05-18 20:08
Yes it's very strange which is why I'm stumped.

I'm running Gentoo Linux 2.6.16r4.

I have opened the ticket with them but atm they just point me to the
process_watchdog.log.

You mentioned you're no longer with textdrive. Could I ask who you're
with?

Thanks again for helping out...

Regards,

Saimon


Ezra Zygmuntowicz wrote:
> Hi !
>
> On May 18, 2006, at 9:55 AM, Saimon Moore wrote:
>
>>
>> a.name }.join('
>> details.
>> (STI)
>> production mode) and it appears rock solid. The fcgi process has a
>>
>> In short I'm stumped as to why this is behaving so differently on
>> textdrive and I'm not really sure what I can do to optimize this.
>>
>> I'd appreciate any advice.
>>
>> Thanks,
>>
>> Saimon
>
>
>
> Saimon-
>
> 	Yeah that sounds a bit fishy when it works fine on your local box
> but not on txtdrv server. What OS are you running locally? I don't
> think I have any clear cut answers for you right off the bat here.
> Since it seems like a textdrive problem you should probably file a
> support ticket there detailing how it works locally but doesn't work
> on their servers.
>
> 	I no longer use textdrive so I can't say what the issue might be.
> Hope you get it figured out though.
>
> Cheers-
> -Ezra
6076c22b65b36f5d75c30bdcfb2fda85?d=identicon&s=25 Ezra Zygmuntowicz (Guest)
on 2006-05-18 20:47
(Received via mailing list)
On May 18, 2006, at 11:08 AM, Saimon Moore wrote:

> Thanks again for helping out...
>
> Regards,
>
> Saimon
>

Saimon-

	I host a bunch of rails stuff on Xen based VPS's with http://
rimuhosting.com  It is so much nicer to have your own separate
environment to run your rails apps in without anyone else killing
your processes and such.

	I have written a tutorial a while ago about setting up a debian vps
from scratch here:  http://brainspl.at/rails_stack.html

	If you don't want to be your own sysadmin I also like PlanetArgon
for shared hosting.

-Ezra
982c3f128bb78678033c0bf8c0e226bc?d=identicon&s=25 Saimon Moore (saimon)
on 2006-05-18 23:15
I've had a look at rimuhosting.com. Seems quite interesting. As this
application is for a client of mine and at the moment I'm using my own
personal sharing plan to host it, I'm probably going to recommend a
hosting change if things don't pan out with textdrive (I was going to do
that sooner or later) so I'll be evaluating it and Planet Argon too.

Nice tutorial...

Thanks for the info...

Ezra Zygmuntowicz wrote:
> On May 18, 2006, at 11:08 AM, Saimon Moore wrote:
>
>> Thanks again for helping out...
>>
>> Regards,
>>
>> Saimon
>>
>
> Saimon-
>
> 	I host a bunch of rails stuff on Xen based VPS's with http://
> rimuhosting.com  It is so much nicer to have your own separate
> environment to run your rails apps in without anyone else killing
> your processes and such.
>
> 	I have written a tutorial a while ago about setting up a debian vps
> from scratch here:  http://brainspl.at/rails_stack.html
>
> 	If you don't want to be your own sysadmin I also like PlanetArgon
> for shared hosting.
>
> -Ezra
5c0f4e71d9d0b71b8b60a2693df1571a?d=identicon&s=25 Deirdre Saoirse Moen (Guest)
on 2006-05-26 00:45
(Received via mailing list)
On May 18, 2006, at 9:55 AM, Saimon Moore wrote:

> I have suspected Rmagick as a possible offender but after further
> tests
> I have rejected it as a possibility. (Images are processed via rmagick
> just the once and thereafter they are cached on the browser)

FYI, historically Rmagick has been the source of many issues on
TextDrive because usage can go through the roof. Also, many TxD
servers are 64-bit, which would have different memory usage patterns
than you may expect.

--
_Deirdre                                             http://deirdre.net
This topic is locked and can not be replied to.