Runaway FCGI Processes with Debian, Apache


#1

Hi all.

There is an issue with Rails, FastCGI and Apache2 on
Debian that I’ve been dealing with since I started
Rails development in late 2005. I’ve developed
various workarounds but have not been able to solve
the problem at the root.

Every so often, for reasons I’ve never been able to
determine, a load of dispatch.fcgi processes get
spawned and take over my whole server, effectively
rendering the web site useless. The only solution is
to manually kill the processes.

I’ve seen other posts about this issue but I’ve never
seen a real solution – only workarounds like having
a script that periodically searches for these
dispatch.fcgi processes and kills them.

Our web site is currently only in a test or beta
stage, so the issue isn’t too bad – but soon we’ll be
attracting a lot more traffic (I hope), and I don’t
feel comfortable not knowing why my server is killed
by dispatch.fcgi processes.

If anyone can provide information about this, I’ll
collate it and write it up properly and make it
accessible. My questions are:

  1. What’s the cause? Is it a problem with the FCGI
    module, Apache, or the Rails dispatcher?

  2. Does the problem only exist for certain Rails
    versions, or for Rails on Debian, or all Rails
    versions on all *nix platforms?

  3. If you’ve never experienced any problem like this,
    and you use Apache 2 and FCGI with Rails, what version
    of Rails do you use, and what *nix distro?

  4. Finally, what’s a real solution to the problem?
    I’ll accept having to change to a different web
    server, a different version of Rails, or a different
    distro if I absolutely know it will solve my problem.

Many thanks in advance for any info.

Matt J.


Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com


#2

I’m running Debian 3.1 and had exactly the same issues. Apache2 and
mod_fastcgi doesn’t play well together. I’ve switched to mod_fcgid and
it
works fine so far.

Kent.


#3

Nice one. I’ve changed to fcgid; I’ll let it run for
a few days and see how it fares. If it works I’ll add
a note to the Rails wiki and other documentation.

Thanks

— Kent S. removed_email_address@domain.invalid wrote:

Hi all.
to
a script that periodically searches for these

versions on all *nix platforms?
server, a different version of Rails, or a



Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com


#4

I’ve not come up with a good solution to this either.

I don’t get the “randomly spawning” processes, but it seems that after a
web
restart some FCGI dispatchers are left hanging around. Over time they
can
build up and take over the machine.

I have 3 production sites on Apache 2 / (insert unix here Debian | FBSD)
/
FCGI and experience the problem on all 3.

What’s a clean way to handle this? I don’t want to have to resort to
writing
a script to reap the processes ala dreamhost. That would suck.


#5

Matt,

How has this gone for you? I am using Lighttpd 1.4.11 on debian
testing and I get the same behaviour you were talking about. I’m
manually spawning my FCGI instances and running them on a socket so I
would have thought that any adaptive spawning would have been out of
the question… but I guess that’s just one of the reasons why this
problem is so irritating :slight_smile:

If mod_fcgid under apache is the way to go I’m keen to hear it.

Cheers,
Dan


Dan S.
www.peoplehub.com.au
www.dansketcher.com


#6

Try mod_fcgid:
http://datanoise.com/articles/2006/04/06/apache2-with-mod_fcgid

On 4/10/06, subimage interactive removed_email_address@domain.invalid wrote:

a script to reap the processes ala dreamhost. That would suck.


Rails mailing list
removed_email_address@domain.invalid
http://lists.rubyonrails.org/mailman/listinfo/rails


Kent

http://www.datanoise.com


#7

On 4/11/06, subimage interactive removed_email_address@domain.invalid wrote:

I’ve not come up with a good solution to this either.

I don’t get the “randomly spawning” processes, but it seems that after a web
restart some FCGI dispatchers are left hanging around. Over time they can
build up and take over the machine.

That’s the reason I stopped letting lighttpd do the spawning and set
up static fcgi processes like this:
http://jamis.jamisbuck.org/articles/2006/02/11/tip-textdrive-and-lighttpd
The difference for me is that I moved from a FreeBSD server to a
Debian one and this problem started. I think I’ll have a crack at
Mongrel once I work out how Capistrano will interact with it (can’t
find much info on this).

Cheers,
Dan


Dan S.
www.peoplehub.com.au
www.dansketcher.com


#8
  • Dan S. (removed_email_address@domain.invalid) [060410 18:30]:

How has this gone for you? I am using Lighttpd 1.4.11 on debian
testing and I get the same behaviour you were talking about. I’m
manually spawning my FCGI instances and running them on a socket so I
would have thought that any adaptive spawning would have been out of
the question… but I guess that’s just one of the reasons why this
problem is so irritating :slight_smile:

If mod_fcgid under apache is the way to go I’m keen to hear it.

FWIW, on Debian, using Apache2 to front lighttpd with lighttpd
delegating to fcgi Rails has worked well for us (i.e., zero problems).
Using Apache{1,2}, on Debian, to serve FCGI Rails directly has been a
constant headache and I do not recommend it at all. Intermittent 500
errors, “Application Error (Rails)” at random times, buildup of fcgi
processes, and eventual swamping of the machine were consistently seen.

Rick

http://www.rickbradley.com MUPRN: 702
| him speak before. Too
random email haiku | bad. .understandable of
| course. but too bad.


#9

I should clarify that I still get zombie processes even when I have
static fcgi processes on Debian Etch - on FreeBSD 5.4/6.0 I had no
problems.


Dan S.
www.peoplehub.com.au
www.dansketcher.com


#10

I agree wholeheartedly. I had a problem with random fcgi processes
before I switched and it has disappeared completely. The one gotcha I
encountered was to set production mode in environment.rb b/c I couldn’t
get the environment variables to pass, but that is no biggie really.

Nicholas P. Mueller


#11

I’ve also set static fcgi processes and I have to force into production
mode, but I still get zombie fcgi’s from time to time. It’s almost as if
they don’t get shut down properly on an apache restart.

This is a little off topic, but related - how are you all restarting
your
app when pushing new code to production? I’m under the impression that
you
have to actually restart apache in order for the changes to take effect.


#12

I’ve found a few posts that might help some having this problem -

http://www.fastcgi.com/archives/fastcgi-developers/2005-May/003720.html
which links to
http://rubyists.com/articles/2005/05/03/spawn-fcgi-in-the-foreground

There is a ticket to get it integrated directly into Lighttpd in
http://trac.lighttpd.net/trac/ticket/216 - although from what I gather
in the patch it’s more of fastcgi rather than lighty

Doesn’t help me though because I can’t get the right build environment
on debian testing for some reason - probably because I am a debian
noob. I wanted to have a crack at the changes from
http://trac.lighttpd.net/trac/ticket/554 to see if it fixed anything
but I guess I’ll just have to accept it for now…

Does anyone have a good script to kill off the extra zombies?


Dan S.
www.peoplehub.com.au
www.dansketcher.com