Is there a reason that there has not been an attempt to directly call
CGI programs (compiled CGI, perl CGI, whatever) in nginx?
I would love to take out proxying to apache if I could.
Is there a blocking nature in CGI or something else that makes it
impossible to function, or has there been no interest? I have a
handful of apps I have to support (bugzilla, mailman, etc.) that I
have to proxy to apache right now.
I’d be willing to put some money down for it, too. (and mod_svn !)
On Fri, Feb 06, 2009 at 02:55:25PM -0800, mike wrote:
have to proxy to apache right now.
I’d be willing to put some money down for it, too. (and mod_svn !)
nginx is not general purpose server, it’s rather highload server.
CGI is not compatible with highload: if you run Apache/CGI, then CGI
will became bottleneck much earlier than Apache.
There are two ways to implement CGI inside nginx:
simple one: just fork()ing worker process that has received a request
for CGI and exec() a CGI program. It’s simple enough, but has a lot
of overhead. Besides CGI programs will run with worker privilege only.
complex way: to run a special CGI manager (probably with root
privilege
as master process) and to pass it requests/sockets using Unix domain
sockets.
Then the manager can fork/exec CGI programs with required privileges
and with minimal overhead to nginx workers.
The second way will require some time to program, but the outcome will
be much similar like just proxying to mini_httpd
( mini_httpd ).
BTW, it seems that using Apache with several worker processes (2-5) for
bugzilla, mailman, etc. will not consume much CPU/memory: look top.
On Sat, Feb 07, 2009 at 02:33:09AM -0800, mike wrote:
of overhead. Besides CGI programs will run with worker privilege only.
and it requires the CGI program to be modified, yeah?
No.
into a small httpd?
No, the special CGI manager is a nginx process that fork()ed by master
nginx
process (like worker processes). The communication protocol between
workers and the manager is not HTTP: a worker starts to handle a
request,
then it sees that the request should be handled by CGI, something like
simple one: just fork()ing worker process that has received a
request
for CGI and exec() a CGI program. It’s simple enough, but has a lot
of overhead. Besides CGI programs will run with worker privilege
only.
and it requires the CGI program to be modified, yeah?
No.
Got any urls for examples then?
will
request,
BTW, it seems that using Apache with several worker processes
Sure. So it isn’t impossible if someone else was hired to do it. I was
wondering if there was a technical reason for it.
On a different note, I’ve paid someone to implement mod_auth_gss (more
or less) into nginx. So it might support kerberos tickets and single
sign on soon (at least SPNEGO)
nginx is not general purpose server, it’s rather highload server.
CGI is not compatible with highload: if you run Apache/CGI, then CGI
will became bottleneck much earlier than Apache.
There are two ways to implement CGI inside nginx:
simple one: just fork()ing worker process that has received a request
for CGI and exec() a CGI program. It’s simple enough, but has a lot
of overhead. Besides CGI programs will run with worker privilege only.
and it requires the CGI program to be modified, yeah?
complex way: to run a special CGI manager (probably with root privilege
as master process) and to pass it requests/sockets using Unix domain sockets.
Then the manager can fork/exec CGI programs with required privileges
and with minimal overhead to nginx workers.
The second way will require some time to program, but the outcome will
be much similar like just proxying to mini_httpd
( mini_httpd ).
this is like starting out with something like php-fpm, and morphing
into a small httpd?
BTW, it seems that using Apache with several worker processes (2-5) for
bugzilla, mailman, etc. will not consume much CPU/memory: look top.
I’m just looking at it from simplifying the system administration.
I know it is not the most performant, those tools are not my preferred
ones, I just have to support hosting them right now…
nginx is not general purpose server, it’s rather highload server.
CGI is not compatible with highload: if you run Apache/CGI, then CGI
will became bottleneck much earlier than Apache.
This is not a problem from the Nginx point of view.
Nginx should however put a limit on the number of concurrent CGI
requests.
It can use a queue to store outgoing request, and if the queue grows too
much return a 503 Service Unavailable, setting Retry-After to a
reasonable (computed?) value.
No browser support 503 response, however.
There are two ways to implement CGI inside nginx:
simple one: just fork()ing worker process that has received a request
for CGI and exec() a CGI program. It’s simple enough, but has a lot
of overhead. Besides CGI programs will run with worker privilege only.
Nginx could use seteuid, instead of setuid.
So that it can reacquire root privileges.
As for CGI support, some time ago I was trying to implement it.
The idea was to use unix domain sockets (socketpair), and reusing the
http_upstream module.
However I gave up, a lot of code for connection/upstream setup must be
rewritten, and I hate to write boiler plate code; and expecially having
to maintain it :).
In Nginx one can also easily close all current opened file descriptors,
in child process; Nginx keeps all opened connection in the
ngx_cycle->free_connections variable so one can write a closefrom
function.
or is there something fundamental missing from that? I’m running over a
hundred of those at the moment and they seem to do their job (talk
fastcgi on one end and cgi on the other), no cgi code modification
required,
pathinfo support, what’s not to like?
On Sat, Feb 07, 2009 at 09:38:41PM +0100, Manlio P. wrote:
It can use a queue to store outgoing request, and if the queue grows too
much return a 503 Service Unavailable, setting Retry-After to a
reasonable (computed?) value.
No browser support 503 response, however.
I mean that using CGI as main response handler on highload site is
useless.
CGI may be used for lowload tasks: monitoring, administration, etc.
those
can be easly handled by Apache or by proxying to Apache.
There are two ways to implement CGI inside nginx:
simple one: just fork()ing worker process that has received a request
for CGI and exec() a CGI program. It’s simple enough, but has a lot
of overhead. Besides CGI programs will run with worker privilege only.
Nginx could use seteuid, instead of setuid.
So that it can reacquire root privileges.
Yes, but it’s not so safe as setuid().
As for CGI support, some time ago I was trying to implement it.
The idea was to use unix domain sockets (socketpair), and reusing the
http_upstream module.
However I gave up, a lot of code for connection/upstream setup must be
rewritten, and I hate to write boiler plate code; and expecially having
to maintain it :).
Yes, CGI module should use modified upstream module.
In Nginx one can also easily close all current opened file descriptors,
in child process; Nginx keeps all opened connection in the
ngx_cycle->free_connections variable so one can write a closefrom function.
This can be resolved using fcntl(F_SETFD, FD_CLOEXEC) too.
I’m currently testing this with PHPmotion and it works perfectly using
spawn-fcgi to get it started. Now I need to update the code to get Nginx
upload progress bar in there as the one that comes with the scrip is
meant for (cr)apache.
I was wondering, Grzegorz, if there is some “algorithm” as to how to
determine the optimal number of children to run.
bugzilla, mailman, etc. will not consume much CPU/memory: look top.
Sure. So it isn’t impossible if someone else was hired to do it. I was
wondering if there was a technical reason for it.
No, there is no technical reason that does not allow to implement CGI
in nginx, this is wasting time only (from my point of view :).
On Sat, Feb 07, 2009 at 09:50:47PM -0500, Jim O. wrote:
I’m currently testing this with PHPmotion and it works perfectly using spawn-fcgi to get it started. Now I need to update the code to get Nginx upload progress bar in there as the one that comes with the scrip is meant for (cr)apache.
I was wondering, Grzegorz, if there is some “algorithm” as to how to determine the optimal number of children to run.
Basically, the number of concurrent CGI requests you wish to serve and
then some (a single fcgiwrap does not multiplex multiple requests yet,
sorry). It’s actually better to have a few slack ones (they are really very cheap, especially after the first one – double-digit
_kilo_bytes).
On Sun, Feb 08, 2009 at 01:19:12AM -0800, mike wrote:
I googled around originally for nginx + CGI stuff (specifically
bugzilla, mailman) and could not find anything.
I’ve been spamming this list with fcgiwrap ever since I wrote it many
months ago, you must’ve been asleep It’s even been on the wiki for
some time now.
This might work just fine! I will have to try it out. Looks like it is
just like php-fpm, but for CGI?
Roughly. It still needs a process manager of some sort. Have a look at
my website, there’s a very simple launcher script there that you might
find suitable.