Spawn-fcgi-1.6.0rc1-r16 prerelease]

Server NGINX
1

For those people who, for various reasons, can’t or don’t want to
compile PHP with php-fpm (or for anyone who needs to spawn FCGI
processes in general), spawn-fcgi has been split out of Lighttpd into
its own project (message below).

Regards,
Cliff

-------- Forwarded Message --------
From: Stefan Bühler [email protected]
To: [email protected]
Subject: [lighttpd-announce] spawn-fcgi-1.6.0rc1-r16 prerelease
Date: Thu, 19 Feb 2009 00:42:13 +0100

Prerelease of spawn-fcgi 1.6.0 (rc1-r16)

spawn-fcgi extracted into own project

There are several versions of spawn-fcgi (one in lighttpd 1.4.x, one in
1.5.x
and one in cherokee), which makes packaging very hard (e.g.: debian
packages
spawn-fcgi from lighty as “spawn-fcgi.lighttpd” - which makes using it
more
complicated as your scripts are probably using “spawn-fcgi”…).
Additionally, keeping the sources and bug fixes synchronized is just too
much
work (ok, the one in cherokee isn’t our problem :slight_smile: ).
Therefore we decided to split spawn-fcgi into a new project which will
start
with version 1.6.0 (and will be independent of lighttpd releases).

We plan to remove spawn-fcgi from lighttpd-1.4.x after the release of
lighttpd-1.4.22.

You can find the new project at
http://redmine.lighttpd.net/projects/spawn-fcgi/wiki

In case you would like to test a spawn-fcgi version with more features
(options for chown the socket), see here:
http://cgit.stbuehler.de/gitosis/spawn-fcgi/about/ (needs glib)

Important changes

  • Removed limits for @-C phpchildren@ option, which now justs sets
    @PHP_FCGI_CHILDREN@ if @phpchildren >= 0@ (default: ommited); php’s
    default
    is @PHP_FCGI_CHILDREN = 0@, which means that the master process is the
    worker
    process. (not the same as @PHP_FCGI_CHILDREN = 1@, which results in one
    master watching its child). Instead of using the @-C@ option you can now
    just
    set the environment var before invoking spawn-fcgi.
  • Modified the log messages format

Downloads

Changes

  • Separated spawn-fcgi from lighttpd
  • Remove limits für php children; per default the PHP_FCGI_CHILDREN var
    is not
    changed (php defaults to no children, one worker)
  • Modified the log messages format (more details on errors, no source
    line)
  • Only try to connect to unix socket (not tcp) before spawning (fixes
    again
    #1575)
  • Only disconnect from terminal in fork mode (keep stderr/stdout open in
    nofork mode)
  • Allow numerical user and group ids for -u/-g (fixes #1141)
  • Ignore pid-file option in no-fork mode (instead of producing empty
    file)
  • Fix error handling for unix-socket-connect test
2

Yeah I saw that and I wanted to ask why anyone would use it after
using php-fpm…

Seems like spawn-fcgi is designed only for PHP … fcgiwrap is aiming
for general purpose too… I would only see a benefit for spawn-fcgi
if it was useful for other things than PHP now that php-fpm is
there…

3

YMMV, but I had a lot of trouble using spawn-fcgi + php. Processes
would stop responding to requests after a while for no reason. Turns out
PHP has a hard limit on the number of requests it can take before
re-spawning a child and spawn-fcgi didn’t honor that limit, passing
requests to quitting children.

I posted this information to the list and forums but was ignored, which
expidited the move to nginx. I’m now running php-cgi with the -b switch
on a number of production servers without issue (php-fpm wouldn’t
compile for me at the time).

Cliff W. wrote:

To: [email protected]
complicated as your scripts are probably using “spawn-fcgi”…).

process. (not the same as @PHP_FCGI_CHILDREN = 1@, which results in one
MD5: 19099d04b133222329f1f6eed12b6398
changed (php defaults to no children, one worker)

–

Phillip B Oldham
The Activity People
[email protected] mailto:[email protected]

Policies

This e-mail and its attachments are intended for the above named
recipient(s) only and may be confidential. If they have come to you in
error, please reply to this e-mail and highlight the error. No action
should be taken regarding content, nor must you copy or show them to
anyone.

This e-mail has been created in the knowledge that Internet e-mail is
not a 100% secure communications medium, and we have taken steps to
ensure that this e-mail and attachments are free from any virus. We must
advise that in keeping with good computing practice the recipient should
ensure they are completely virus free, and that you understand and
observe the lack of security when e-mailing us.

4

That’s what I wound up doing. Something akin to exporting the 2
environment variables and then “sudo -u $user php-cgi -b $port”

Seemed to work great, until I found php-fpm (I had an idea sort of
like php-fpm and then someone pointed it to me - and bam… I fell in
love)

On Thu, Feb 19, 2009 at 1:40 AM, Phillip B Oldham

5

On Wed, 2009-02-18 at 22:25 -0800, mike wrote:

Yeah I saw that and I wanted to ask why anyone would use it after
using php-fpm…

I’ve had zero issues with spawn-fcgi over the last couple years. I
can’t speak for anyone else, but the appeal of maintaining a custom PHP
build is of zero interest to me.

Seems like spawn-fcgi is designed only for PHP … fcgiwrap is aiming
for general purpose too… I would only see a benefit for spawn-fcgi
if it was useful for other things than PHP now that php-fpm is
there…

AFAIK, spawn-fcgi can be used to launch any FCGI app. I rarely use
FCGI except for PHP, but on Lighty spawn-fcgi is the de facto standard
for launching a FCGI, so don’t see how it could be PHP-specific…
unless you are asserting Lighty can only host PHP FCGI apps?

Cliff

6

My qualm was having to write a custom script to manage spawn-fcgi
pools and it ignored the max requests environment variable (that must
be fixed by now)

Yes the only drawback with php-fpm is the custom build but if you want
an optimized binary with all the stuff you want and not the stuff you
don’t want that’s what of comes down to anyway.

I tried looking for an example of spawn-fcgi for something other than
php (very quickly though) and every example was for that…

7

As a lighty dev, I feel urged to correct some misinformation presented
in this thread.

mike wrote:

Seems like spawn-fcgi is designed only for PHP … fcgiwrap is aiming
for general purpose too… I would only see a benefit for spawn-fcgi
if it was useful for other things than PHP now that php-fpm is
there…

As Cliff W. already correctly wrote:
spawn-fcgi is not designed only for PHP, it can spawn any FastCGI
application.
You can use it with rails, python, java or whatever.
That’s why it is called spawn-fcgi and not spawn-php.

Phillip B Oldham write:

YMMV, but I had a lot of trouble using spawn-fcgi + php. Processes
would stop responding to requests after a while for no reason. Turns out
PHP has a hard limit on the number of requests it can take before
re-spawning a child and spawn-fcgi didn’t honor that limit, passing
requests to quitting children.

Requests do NOT go through spawn-fcgi. It is only used to set up the
FastCGI environment (file descriptors for example) and hand over control
to the application.
Therefor it is impossible that spawn-fcgi is to blame for PHP not
getting requests.
PHP can spawn multiple worker children which (to prevent leaks) are
restarted after a certain, configurable, number of requests. That is
true. But it is the PHP master process that hands the requests over to
the children and it of course handles the restarts.
The problem you experienced clearly must be originating somewhere else.

8

be fixed by now)
I had the same issue.

On Wed, 2009-02-18 at 22:25 -0800, mike wrote:

for general purpose too… I would only see a benefit for spawn-fcgi
if it was useful for other things than PHP now that php-fpm is
there…

AFAIK, spawn-fcgi can be used to launch any FCGI app. I rarely use
FCGI except for PHP, but on Lighty spawn-fcgi is the de facto
standard
for launching a FCGI, so don’t see how it could be PHP-specific…
unless you are asserting Lighty can only host PHP FCGI apps?

But it won’t launch more than one instance of anything beside php. The
-C
flag only works for php - unless that is changed with the new version as
well. Using it with fcgiwrap is very limiting.

9

mike wrote:

However you need to launch one spawn-fcgi per thread you want, it does
not pool for you except with php fastcgi.

I think you are confusing threads with processes.
Threads need to be managed by the FastCGI processes, there’s nothing
spawn-fcgi can or should do in this regard.
You can spawn multiple processes with spawn-fcgi, binding them to
multiple addresses or, like in the case of PHP, let the FastCGI
application do that (if it is capable of doing so).
I don’t see any problem here.

10

On Sun, Mar 8, 2009 at 10:10 AM, Icy – [email protected] wrote:

As Cliff W. already correctly wrote:
spawn-fcgi is not designed only for PHP, it can spawn any FastCGI
application.
You can use it with rails, python, java or whatever.
That’s why it is called spawn-fcgi and not spawn-php.

However you need to launch one spawn-fcgi per thread you want, it does
not pool for you except with php fastcgi.

11

So I can launch with a single command line

spawn-fcgi -some -options some-other-cgi-stuff-than-php here

and it will allow more than one concurrent connection?

I think the latest version might have mentioned supporting something,
but I’d be looking at leveraging it to manage fcgiwrap, and I don’t
want to have to launch one spawn-fcgi command for every individual
amount of requests I intend.

i.e.

I want a pool of fcgiwrap resources available using a single command
line for a specific website. I want to give it a specific user/group
ID to run as. (or anything else like fcgiwrap, basically I want to run
non-PHP CGI scripts (perl, compiled CGI) from nginx)

It sounds like the latest updates to spawn-fcgi might help this become
a reality. Did you say you maintained the project? Have any command
line options that I can try to make this happen? I want to be able to
support say, 10 concurrent requests. In PHP that would translate
roughly to 10 php children (-C 10 I think?)

Do I have to do this 10 times:
spawn-fcgi -a -b -c /usr/local/bin/fcgiwrap

Or can I launch it just once with some parameters?

12

The biggest problem is still that spawn-fcgi is known to crash and hang

  • separate issue, I know. There are lots of references on it. Consult
    Professor Google and you’ll find many. I’d start with
    spawn-fcgi crash - Google Search and
    spawn-fcgi hang - Google Search. I haven’t tried using it
    in a year or more. Perhaps it’s improved in that regard but I’m
    reluctant to use it in a production environment when php-fpm is just
    about bullet-proof as it gets. Of course that assumes that development
    of php-fpm continues as it’s presently a one man show AFAIK. Now I’m
    aware that this doesn’t happen on every system but it happens a lot and
    it has happened to me.

Aside from that, even though fcgiwrap is capable of running several
processes simultaneously, spawn-fcgi will only start one. I may be wrong
but I believe that’s an inherent limitation of spawn-fcgi’s command
options.

13

On nie, mar 08, 2009 at 12:17:57 -0800, mike wrote:

So I can launch with a single command line

spawn-fcgi -some -options some-other-cgi-stuff-than-php here

and it will allow more than one concurrent connection?

Depends on what you run. It’s none of spawn-fcgi’s business, actually.
The FastCGI server you run may be a preforked PHP intepreter, a
single-process fcgiwrap, a multithreaded Python application or just
about anything else. It doesn’t really have to speak FastCGI, either
(spawn-fcgi definitely does not care).

By the time your application is ready to accept its first request
spawn-fcgi is long gone and forgotten (it doesn’t remain in memory to
manage the spawned process).

When I last looked at it, spawn-fcgi could only spawn a single process,
but that process is then free to do whatever it wishes, including
starting a pool of child processes and managing them (a’la php-fcgi or
new fcgiwrap).

Do I have to do this 10 times:
spawn-fcgi -a -b -c /usr/local/bin/fcgiwrap

Or can I launch it just once with some parameters?

spawn-fcgi -s foo.sock – /usr/local/bin/fcgiwrap -c 10

This is a fcgiwrap feature, spawn-fcgi has nothing to do with it, but
there’s nothing preventing spawn-fcgi from implementing such a prefork
itself.

Best regards,
Grzegorz N.

14

mike wrote:

So I can launch with a single command line

spawn-fcgi -some -options some-other-cgi-stuff-than-php here

and it will allow more than one concurrent connection?

That is up to the application, not spawn-fcgi.
It could do threading, have multiple worker processes or whatnot.

I think the latest version might have mentioned supporting something,
but I’d be looking at leveraging it to manage fcgiwrap, and I don’t
want to have to launch one spawn-fcgi command for every individual
amount of requests I intend.

i.e.

I want a pool of fcgiwrap resources available using a single command
line for a specific website. I want to give it a specific user/group
ID to run as. (or anything else like fcgiwrap, basically I want to run
non-PHP CGI scripts (perl, compiled CGI) from nginx)

It sounds like the latest updates to spawn-fcgi might help this become
a reality. Did you say you maintained the project? Have any command
line options that I can try to make this happen? I want to be able to
support say, 10 concurrent requests. In PHP that would translate
roughly to 10 php children (-C 10 I think?)

Do I have to do this 10 times:
spawn-fcgi -a -b -c /usr/local/bin/fcgiwrap

Or can I launch it just once with some parameters?

I have never used fcgiwrap and in fact just discovered it. As I
understand it, it uses fcgi to then spawn cgi processes? Why not code
the application as a FastCGI one? That would be way faster in terms of
performance.
The only reason you would need that is to either run CGI apps on a
remote host or to overcome the lack of cgi support in nginx.

Jim O. wrote:

The biggest problem is still that spawn-fcgi is known to crash and hang

  • separate issue, I know. There are lots of references on it. Consult
    Professor Google and you’ll find many. I’d start with
    spawn-fcgi crash - Google Search and
    spawn-fcgi hang - Google Search. I haven’t tried using it
    in a year or more. Perhaps it’s improved in that regard but I’m
    reluctant to use it in a production environment when php-fpm is just
    about bullet-proof as it gets. Of course that assumes that development
    of php-fpm continues as it’s presently a one man show AFAIK. Now I’m
    aware that this doesn’t happen on every system but it happens a lot and
    it has happened to me.

And all you find via google are invalid bugreports. Saying something can
be found on the internet doesn’t make it true.
Again I think you guys completely misunderstand the role of spawn-fcgi.
It sets up some sockets and spawns a processes. That’s basically it.
It is in NO way involved with handling the requests. If something
crashes or hangs then that’s the fault of the application. PHP in this
case (and it has been known to do so in past releases).
This is not spawn-fcgi’s fault. After spawning the application,
spawn-fcgi is no more around normally. How can something that is not
running cause such problems?

Aside from that, even though fcgiwrap is capable of running several
processes simultaneously, spawn-fcgi will only start one. I may be wrong
but I believe that’s an inherent limitation of spawn-fcgi’s command
options.

You can of course (as I said earlier) start multiple processes by
calling spawn-fcgi multiple times (with different command options for
the address).
But even a single process could handle many concurrent requests (see
lighty).
There are many ways (like I wrote before too) to do so: threads, child
processes, multiplexing…
Telling spawn-fcgi to spawn like 10 processes is not possible because
how should it know which addresses to bind to?

15

On Sun, Mar 8, 2009 at 2:10 PM, Grzegorz N.
[email protected] wrote:

spawn-fcgi -s foo.sock – /usr/local/bin/fcgiwrap -c 10

This is a fcgiwrap feature, spawn-fcgi has nothing to do with it, but
there’s nothing preventing spawn-fcgi from implementing such a prefork
itself.

Cool. Okay, this is starting to move along then.

This would be equivalent then of PHP_FCGI_CHILDREN or whatever, more
or less, yes?

I would run this just once and I would have essentially 10 concurrent
socket connections available?

16

On Sun, Mar 8, 2009 at 2:02 PM, Icy – [email protected] wrote:

That is up to the application, not spawn-fcgi.
It could do threading, have multiple worker processes or whatnot.

Grzegorz knows best. Looks like he has replied.

I have never used fcgiwrap and in fact just discovered it. As I
understand it, it uses fcgi to then spawn cgi processes? Why not code
the application as a FastCGI one? That would be way faster in terms of
performance.

Oh, I prefer PHP (and I guess anything else with FastCGI) - however, I
have to support bugzilla, mailman and other older apps right now. I
don’t like it, but there are no good replacements out there that my
customers have found yet (I was thinking of trying to rewrite mailman
in PHP in fact)

The only reason you would need that is to either run CGI apps on a
remote host or to overcome the lack of cgi support in nginx.

Correct - nginx doesn’t natively support it.

And all you find via google are invalid bugreports. Saying something can
be found on the internet doesn’t make it true.

spawn-fcgi was not properly recycling php engines (i.e.
PHP_MAX_FCGI_REQUESTS or whatever) when i was using it, i was able to
validate it. i wound up going back to a custom php -b solution until i
found php-fpm. so there -are- bugs.

You can of course (as I said earlier) start multiple processes by
calling spawn-fcgi multiple times (with different command options for
the address).
But even a single process could handle many concurrent requests (see
lighty).
There are many ways (like I wrote before too) to do so: threads, child
processes, multiplexing…
Telling spawn-fcgi to spawn like 10 processes is not possible because
how should it know which addresses to bind to?

I have no clue.

Can Grzegorz get in here and you two can hash out how it works? :slight_smile:

17

mike wrote:

spawn-fcgi was not properly recycling php engines (i.e.
PHP_MAX_FCGI_REQUESTS or whatever) when i was using it, i was able to
validate it. i wound up going back to a custom php -b solution until i
found php-fpm. so there -are- bugs.

How can spawn-fcgi do something not properly when it doesn’t do it at
all?
I don’t know how many times I will have to write this but it isn’t
spawn-fcgi’s job to recycle the PHP processes, handle requests or
anything similar.
You could not have verified this because it doesn’t even happen!
It spawns the process and that’s all.

18

On nie, mar 08, 2009 at 02:18:55 -0800, mike wrote:

This would be equivalent then of PHP_FCGI_CHILDREN or whatever, more
or less, yes?

I would run this just once and I would have essentially 10 concurrent
socket connections available?

Exactly.

Best regards,
Grzegorz N.

19

On nie, mar 08, 2009 at 11:02:02 +0100, Icy – wrote:

I have never used fcgiwrap and in fact just discovered it. As I
understand it, it uses fcgi to then spawn cgi processes? Why not code
the application as a FastCGI one? That would be way faster in terms of
performance.
The only reason you would need that is to either run CGI apps on a
remote host or to overcome the lack of cgi support in nginx.

Well, I wrote it to run unmodified CGI scripts under Nginx. Yes, it’s
basically a FastCGI server that handles requests by running CGI scripts
and passing data to/from them.

And all you find via google are invalid bugreports. Saying something can
be found on the internet doesn’t make it true.
Again I think you guys completely misunderstand the role of spawn-fcgi.
It sets up some sockets and spawns a processes. That’s basically it.
It is in NO way involved with handling the requests. If something
crashes or hangs then that’s the fault of the application. PHP in this
case (and it has been known to do so in past releases).
This is not spawn-fcgi’s fault. After spawning the application,
spawn-fcgi is no more around normally. How can something that is not
running cause such problems?

QFT.

Aside from that, even though fcgiwrap is capable of running several
processes simultaneously, spawn-fcgi will only start one. I may be wrong

… one fcgiwrap, which can then fork into several ones, overcoming that
limitation.

but I believe that’s an inherent limitation of spawn-fcgi’s command
options.

I believe that’s just a missing feature. There’s nothing (except added
complexity of process management) preventing spawn-fcgi from adding a
–prefork= option.

You can of course (as I said earlier) start multiple processes by
calling spawn-fcgi multiple times (with different command options for
the address).

This is not equivalent. If I want multiple identical FastCGI processes,
I always run them on the same socket to completely avoid load balancing
issues.

Telling spawn-fcgi to spawn like 10 processes is not possible because
how should it know which addresses to bind to?

Run them all on the same socket.

Best regards,
Grzegorz N.

20

Grzegorz N. wrote:

Run them all on the same socket.

This could be a problem with tcp sockets :slight_smile:

spawn-fcgi doesn’t care about php engines, CGI processes and whatnot.
If your bug report contains the word PHP, then it certainly isn’t
about spawn-fcgi. There is never a single point in time when both
spawn-fcgi and php-fcgi run side by side (one simply becomes the other).

Very well explained!