Forum: Ruby on Rails Runaway FCGI Processes with Debian, Apache

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
Matt J. (Guest)
on 2006-04-03 20:38
(Received via mailing list)
Hi all.

There is an issue with Rails, FastCGI and Apache2 on
Debian that I've been dealing with since I started
Rails development in late 2005.  I've developed
various workarounds but have not been able to solve
the problem at the root.

Every so often, for reasons I've never been able to
determine, a load of dispatch.fcgi processes get
spawned and take over my whole server, effectively
rendering the web site useless.  The only solution is
to manually kill the processes.

I've seen other posts about this issue but I've never
seen a *real* solution -- only workarounds like having
a script that periodically searches for these
dispatch.fcgi processes and kills them.

Our web site is currently only in a test or beta
stage, so the issue isn't too bad -- but soon we'll be
attracting a lot more traffic (I hope), and I don't
feel comfortable not knowing why my server is killed
by dispatch.fcgi processes.

If anyone can provide information about this, I'll
collate it and write it up properly and make it
accessible.  My questions are:

1) What's the cause?  Is it a problem with the FCGI
module, Apache, or the Rails dispatcher?

2) Does the problem only exist for certain Rails
versions, or for Rails on Debian, or all Rails
versions on all *nix platforms?

3) If you've never experienced any problem like this,
and you use Apache 2 and FCGI with Rails, what version
of Rails do you use, and what *nix distro?

4) Finally, what's a *real* solution to the problem?
I'll accept having to change to a different web
server, a different version of Rails, or a different
distro if I absolutely know it will solve my problem.


Many thanks in advance for any info.


Matt J.

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
Kent S. (Guest)
on 2006-04-03 21:09
(Received via mailing list)
I'm running Debian 3.1 and had exactly the same issues. Apache2 and
mod_fastcgi doesn't play well together. I've switched to mod_fcgid and
it
works fine so far.

Kent.
Matt J. (Guest)
on 2006-04-03 21:49
(Received via mailing list)
Nice one.  I've changed to fcgid; I'll let it run for
a few days and see how it fares.  If it works I'll add
a note to the Rails wiki and other documentation.

Thanks

--- Kent S. <removed_email_address@domain.invalid> wrote:

> > Hi all.
> to
> > a script that periodically searches for these
> >
> > versions on all *nix platforms?
> > server, a different version of Rails, or a
> > __________________________________________________
>
>
__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
Dan S. (Guest)
on 2006-04-11 03:27
(Received via mailing list)
Matt,

How has this gone for you? I am using Lighttpd 1.4.11 on debian
testing and I get the same behaviour you were talking about. I'm
manually spawning my FCGI instances and running them on a socket so I
would have thought that any adaptive spawning would have been out of
the question... but I guess that's just one of the reasons why this
problem is so irritating :)

If mod_fcgid under apache is the way to go I'm keen to hear it.

Cheers,
Dan

--
Dan S.
www.peoplehub.com.au
www.dansketcher.com
seth b. (Guest)
on 2006-04-11 04:28
(Received via mailing list)
I've not come up with a good solution to this either.

I don't get the "randomly spawning" processes, but it seems that after a
web
restart some FCGI dispatchers are left hanging around. Over time they
can
build up and take over the machine.

I have 3 production sites on Apache 2 / (insert unix here Debian | FBSD)
/
FCGI and experience the problem on all 3.

What's a clean way to handle this? I don't want to have to resort to
writing
a script to reap the processes ala dreamhost. That would suck.
Rick B. (Guest)
on 2006-04-11 04:47
(Received via mailing list)
* Dan S. (removed_email_address@domain.invalid) [060410 18:30]:
> How has this gone for you? I am using Lighttpd 1.4.11 on debian
> testing and I get the same behaviour you were talking about. I'm
> manually spawning my FCGI instances and running them on a socket so I
> would have thought that any adaptive spawning would have been out of
> the question... but I guess that's just one of the reasons why this
> problem is so irritating :)
>
> If mod_fcgid under apache is the way to go I'm keen to hear it.

FWIW, on Debian, using Apache2 to front lighttpd with lighttpd
delegating to fcgi Rails has worked well for us (i.e., zero problems).
Using Apache{1,2}, on Debian, to serve FCGI Rails directly has been a
constant headache and I do not recommend it at all.  Intermittent 500
errors, "Application Error (Rails)" at random times, buildup of fcgi
processes, and eventual swamping of the machine were consistently seen.

Rick
--
 http://www.rickbradley.com    MUPRN: 702
                       |  him speak before. Too
   random email haiku  |  bad. .understandable of
                       |  course. but too bad.
Kent S. (Guest)
on 2006-04-11 05:02
(Received via mailing list)
Try mod_fcgid:
http://datanoise.com/articles/2006/04/06/apache2-w...


On 4/10/06, subimage interactive <removed_email_address@domain.invalid> wrote:
> a script to reap the processes ala dreamhost. That would suck.
>
> _______________________________________________
> Rails mailing list
> removed_email_address@domain.invalid
> http://lists.rubyonrails.org/mailman/listinfo/rails
>
>
>


--
Kent
---
http://www.datanoise.com
Dan S. (Guest)
on 2006-04-11 05:14
(Received via mailing list)
On 4/11/06, subimage interactive <removed_email_address@domain.invalid> wrote:
> I've not come up with a good solution to this either.
>
> I don't get the "randomly spawning" processes, but it seems that after a web
> restart some FCGI dispatchers are left hanging around. Over time they can
> build up and take over the machine.
>

That's the reason I stopped letting lighttpd do the spawning and set
up static fcgi processes like this:
http://jamis.jamisbuck.org/articles/2006/02/11/tip...
The difference for me is that I moved from a FreeBSD server to a
Debian one and this problem started. I think I'll have a crack at
Mongrel once I work out how Capistrano will interact with it (can't
find much info on this).

Cheers,
Dan

--
Dan S.
www.peoplehub.com.au
www.dansketcher.com
Nicholas P. Mueller (Guest)
on 2006-04-11 10:09
(Received via mailing list)
I agree wholeheartedly.   I had a problem with random fcgi processes
before I switched and it has disappeared completely.  The one gotcha I
encountered was to set production mode in environment.rb b/c I couldn't
get the environment variables to pass, but that is no biggie really.

Nicholas P. Mueller
Dan S. (Guest)
on 2006-04-11 10:30
(Received via mailing list)
I should clarify that I still get zombie processes even when I have
static fcgi processes on Debian Etch - on FreeBSD 5.4/6.0 I had no
problems.

--
Dan S.
www.peoplehub.com.au
www.dansketcher.com
seth b. (Guest)
on 2006-04-11 10:46
(Received via mailing list)
I've also set static fcgi processes and I have to force into production
mode, but I still get zombie fcgi's from time to time. It's almost as if
they don't get shut down properly on an apache restart.

This is a little off topic, but related - how are you all restarting
your
app when pushing new code to production? I'm under the impression that
you
have to actually restart apache in order for the changes to take effect.
Dan S. (Guest)
on 2006-04-12 05:35
(Received via mailing list)
I've found a few posts that might help some having this problem -

http://www.fastcgi.com/archives/fastcgi-developers...
which links to
http://rubyists.com/articles/2005/05/03/spawn-fcgi...

There is a ticket to get it integrated directly into Lighttpd in
http://trac.lighttpd.net/trac/ticket/216 - although from what I gather
in the patch it's more of fastcgi rather than lighty

Doesn't help me though because I can't get the right build environment
on debian testing for some reason - probably because I am a debian
noob. I wanted to have a crack at the changes from
http://trac.lighttpd.net/trac/ticket/554 to see if it fixed anything
but I guess I'll just have to accept it for now...

Does anyone have a good script to kill off the extra zombies?

--
Dan S.
www.peoplehub.com.au
www.dansketcher.com
This topic is locked and can not be replied to.