Launch FastCGI process?

Hi,

My understand is that nginx doesn’t launch a process, being it a CGI
process
or FastCGI back-end server. I wonder what are the reasons behind this
decision.
I used to use another web server that takes care of launching FastCGI
processes.

The nice thing about it is that the web server checks if a backend
process is
running. If not, it automatically starts it. This means, for whatever
reason if
the backend process crashes, it will be re-launched. Also, the web
server
gets to control how many back-end server processes to start, according
to the conf file. With this managing the processes is really easy,
because
it’s all managed by the web server.

This is a very handy feature. Any chance of having this in nginx?

Thanks,
Jack

jlist9 wrote:

to the conf file. With this managing the processes is really easy, because
it’s all managed by the web server.

This is a very handy feature. Any chance of having this in nginx?

Thanks,
Jack

Thats adding a lot of extra functionality into nginx, and there are many
other daemons which specialize in this functionality. Also, if one of
the children dies badly it could take out nginx itself; not great for
graceful handling of errors since nginx can handle upstreams being down
by returning a custom 50x page.

For daemons which can manage FCGI processes check out Monit (from
tildesoft), Supervisord (python, personal fav.), and God (ruby). I
belive nagios can also do this along-side providing uptime statistics.

Hello jlist9,

Tuesday, October 20, 2009, 3:26:24 PM, you wrote:

Hi,

My understand is that nginx doesn’t launch a process, being it a CGI process
or FastCGI back-end server. I wonder what are the reasons behind this decision.
I used to use another web server that takes care of launching FastCGI processes.

The nice thing about it is that the web server checks if a backend process is
running. If not, it automatically starts it. This means, for whatever reason if
the backend process crashes, it will be re-launched. Also, the web server
gets to control how many back-end server processes to start, according
to the conf file. With this managing the processes is really easy, because
it’s all managed by the web server.

This is a very handy feature. Any chance of having this in nginx?

What if I have nginx running on server A, and many fastcgi backends
running on servers B-Z?

On Tue, Oct 20, 2009 at 01:26:24AM -0700, jlist9 wrote:

to the conf file. With this managing the processes is really easy, because
it’s all managed by the web server.

This is a very handy feature. Any chance of having this in nginx?

nginx will never do it better than simple spawn-fcgi from lighttpd
package,
so I see any advantage of reimplenting spawn-fcgi.

The web server I used before actually has options for both remote and
local
scenarios. For back-end process running locally, the web server can
start
the processes.

On Unix servers, it will also use a Unix domain socket to communicate
with the back-end process, which is faster than a network socket.

I just thought this was very nice since I have one less thing to worry
about. Starting the web server will start the back-end processes and
stopping the web server will stop the back-end processes.

If you’re talking about Apache mod_fastcgi, I agree that its built-in
process management is convenient but if you give up a little
convenience,
you get some rewards in return.

  1. Separation of concerns - let web servers manage requests and
    connections
    and let process managers do process management. It keeps both simpler.
  2. If you use a process manager like supervisord, you can use it to
    manage
    any kind of daemon you may want to run, not just FastCGI daemons.

You could consider using a process manager to spawn both nginx and your
FastCGI daemons. Then you have one place to start and stop everything.

I was actually referring to lighttpd and I think mod_fastcgi does that,
too.
It’s very convenient.

I understand that you can find tools to manage processes. But it’s just
more work, plus in some cases I run a server on a Windows box so many
of the Unix tools are not available. I think in many, maybe most, cases
a site just has both the web server and back-end server running on the
same box. Web server managing the fastcgi/scgi servers would be an
ideal solution.

Also, I think nginx and Python or Java (language of my back-end server)
are very stable so stability isn’t my concern. Ease of setting up and
administration is.

Hi Igor,

In fact, if it’s integrated it is already better :slight_smile:

Although lighttpd has the spawn-fcgi tool, it still integrates this
functionality
into lighttpd itself and this makes setting up simple servers and
managing
them a lot more easier!

As a matter of fact, I don’t quite understand why the replies so far are
mostly
negative about integrating a nice feature into nginx. I think a lot of
lighttpd
users are using this feature and like it a lot. It’s only an option in
lighttpd
and you don’t have to use it. Phillip mentioned that if a child process
is
bad it may bring down the web server. I have never seen this myself.
What I have seen is, if the fcgi process throws an exception and exits,
lighttpd automatically starts another one. How nice! I really hope nginx
does it, too!

Hey,
my take on “why shouldn’t nginx spawn php” below.

On 21 okt 2009, at 07.44, jlist9 wrote:

As a matter of fact, I don’t quite understand why the replies so far
lighttpd automatically starts another one. How nice! I really hope
nginx
does it, too!

That’s because it’s not a web servers job to handle php (or python or
whatever). Php-fpm already does a splendid job of spawning and
maintaining php processes, so I think you should look into that.

Coming from an Apache-world it would probably make sense for nginx to
handle “everything” in your web sphere, but one of the reasons Nginx
is so fast and agile is that its focus is handling http requests and
passing them along, not the other stuff.

On wto, paź 20, 2009 at 01:26:24 -0700, jlist9 wrote:

This is a very handy feature. Any chance of having this in nginx?

We’ve been looking into it for quite some time but eventually decided
against it (we being mostly myself and I am definitely not speaking for
Nginx developers). Nginx supports some weird features like upgrading
binaries on the fly, which doesn’t mesh with managing processes at all.
Or do you want to restart your application servers because you’ve just
made a “zero-downtime” upgrade of Nginx? Not so zero-downtime any more,
is it?

Like fellow posters said, spawning processes by the web server is not
a good idea (especially by a lean and mean server like Nginx). What
we’re using (or at least migrating to) is supervisord[1] for process
management and a custom Nginx module for integration. We have Nginx
communicate with supervisord over XMLRPC to start/shutdown processes as
directed by the load balancer. Supervisord provides process management
way above Nginx could offer, while Nginx provides load data in real
time, so we have exactly as many backends as we need.

If you aim for simply running a pool of backends, look at supervisord,
period.

If you also want to size the pool dynamically, watch this space :slight_smile:
ngx_supervisord isn’t yet available (still waiting for finishing
touches) but consider this a pre-announcement. It’s going to require
patches to the load balancer but if you’re using upstream-fair, the
patch comes with the module source.

Best regards,
Grzegorz N.

  1. http://www.supervisord.org/

jlist9 wrote:

Hi Igor,

In fact, if it’s integrated it is already better :slight_smile:

Although lighttpd has the spawn-fcgi tool, it still integrates this
functionality into lighttpd itself and this makes setting up simple
servers and managing them a lot more easier!

AFAIK lighty just “talks” to the spawnfcgi daemon. Nginx could maybe do
the same with a plugin, but it wouldn’t be managing the fastcgi
processes itself.

As a matter of fact, I don’t quite understand why the replies so far are mostly
negative about integrating a nice feature into nginx. I think a lot of lighttpd
users are using this feature and like it a lot. It’s only an option in lighttpd
and you don’t have to use it. Phillip mentioned that if a child process is
bad it may bring down the web server. I have never seen this myself.

I had this issue consistently while using lighttpd+spawnfcgi+PHP5.
PHP+spawnfcgi would leak memory slowly filling out what was available,
and lighty would balk when a connection couldn’t be made or a response
wasn’t returned correctly. This was a couple of years ago now; things
might’ve changed. I remember plastering the boards and even emailing Jan
directly with no results. Nginx had just made some headlines, so I
switched. We now use it on 6 production servers with PHP, Python, and
Java backends and haven’t had any problems that weren’t down to silly
misconfigurations (on my part - resolved with help from this amazing
maillist!).

What I have seen is, if the fcgi process throws an exception and exits,
lighttpd automatically starts another one. How nice! I really hope nginx
does it, too!

Well, no - nginx doesn’t do anything about it other than capture the
error and move on. That’s the job of the process manager. Its the same
in lighttpd; spawnfcgi manages the fcgi processes, not lighttpd. All
you’re doing is swapping spawnfcgi for supervisord (or God, or
whatever). As Roger mentions, the reason to use a process manager is to
separate concerns. Nginx is concerned with serving content (either from
disk or from an upstream), process managers are concerned with keeping
processes running. Its part of the unix philosophy: “Write programs that
do one thing and do it well.”

Grzegorz N. wrote:

period.

If you also want to size the pool dynamically, watch this space :slight_smile:
ngx_supervisord isn’t yet available (still waiting for finishing
touches) but consider this a pre-announcement. It’s going to require
patches to the load balancer but if you’re using upstream-fair, the
patch comes with the module source.

Wonderful! This sounds just what we’re looking for too. I’ve been
chatting with the maintainers over at supervisord list (Roger et al) and
they’re aiming for another update in the coming weeks that may fix some
issues with FCGI handling, which should make nginx + supervisord + php a
very reliable platform.

or nginx + supervisord + v8cgi ^^ (that i haven’t try that yet…)

Grzegorz N. wrote:

ngx_supervisord cannot manage those processes (the details escape me now
but you have to call it in a slightly different way when you
start/restart/stop it). So just make it a normal ‘program’ and stick
spawn-fcgi in between supervisord and php.

Hopefully the forthcoming updates to supervisord will mitigate this.
Maybe it would be useful to post the problems you foresee on the
supervisord list so the guys can plan further updates for the final 3.0
release?

On śro, paź 21, 2009 at 08:42:08 +0100, Phillip O. wrote:

Wonderful! This sounds just what we’re looking for too. I’ve been
chatting with the maintainers over at supervisord list (Roger et al) and
they’re aiming for another update in the coming weeks that may fix some
issues with FCGI handling, which should make nginx + supervisord + php a
very reliable platform.

One caveat is that if you’re using supervisord’s ‘fcgi-program’ notion,
ngx_supervisord cannot manage those processes (the details escape me now
but you have to call it in a slightly different way when you
start/restart/stop it). So just make it a normal ‘program’ and stick
spawn-fcgi in between supervisord and php.

The processes cannot also be members of a group (due to similar naming
reasons).

These issues are not design limitations, we just didn’t need it, so
it’s just a matter of getting around to it if you do need the features.

Best regards,
Grzegorz N.

I’m not sure if lighttpd is launching fcgi processes directly or it’s
doing
that via spawn-fcgi (I think it’s directly) but it looks the same to me,
in
the sense that I don’t have to worry about the back-end processes.

Please correct me if I’m wrong - my understanding is that spawn-fcgi
is just a launcher. It doesn’t really “manage” any of the process it
launches.
When a fcgi process exits, lighttpd will launch another one. When no
fcgi process is running, lighttpd will start them. When lighttpd quits,
it
will stop all the back-end processes it started. I think it’s fair to
say
that lighttpd is managing these processes.

Yes, I think lighttpd spawns fastcgi process directly. It’s called
“External Spawning” if you use spawn-fcgi.

On śro, paź 21, 2009 at 09:32:06 +0100, Phillip O. wrote:

Hopefully the forthcoming updates to supervisord will mitigate this.
Maybe it would be useful to post the problems you foresee on the
supervisord list so the guys can plan further updates for the final 3.0
release?

Good idea. AFAIK programs must have unique names so allowing a flat
namespace in start/stop etc. (which solves the problem with
ngx_supervisord) won’t introduce any incompatibilities.

Best regards,
Grzegorz N.

Hi Grzegorz,

This is great news. I can’t wait to try it out. Thanks!

Roger

On Tue, Oct 20, 2009 at 11:48 PM, Grzegorz N.

On śro, paź 21, 2009 at 10:24:29 -0700, jlist9 wrote:

Please correct me if I’m wrong - my understanding is that spawn-fcgi
is just a launcher. It doesn’t really “manage” any of the process it launches.

You’re correct and apparently that’s a common misconception about
spawn-fcgi, as in “spawn-fcgi causes timeouts in php” or some such.
spawn-fcgi only exists as a running process for a brief moment before it
becomes (not even spawns) the fastcgi application. See my page at
http://nginx.localdomain.pl/wiki/FcgiWrap – there’s a trivial Perl
script
that duplicates much of spawn-fcgi’s functionality.

AFAIK lighty just “talks” to the spawnfcgi daemon. Nginx could maybe do the
same with a plugin, but it wouldn’t be managing the fastcgi processes
itself.

AFAIK, there’s no “spawnfcgi daemon”. If you want that, use
supervisord. As for nginx talking to supervisord, we’re working on it
(mentioned elsethread).

Best regards,
Grzegorz N.

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs