(fast)cgi and nginx - there has to be a better way

I’ve spent the better part of the day combing through docs and the
mailing list trying to get MovableType working with nginx

it seems the best option is proxying to lighttpd - everything else
seems to involve hacks and copy/pasting scripts off the wiki

For nginx adoption sake, there really has to be a better way.

running php via fcgi is a PITA, and most ways involve using
lighttpd’s fcgi manager
trying to get a fcgi compatible perl app ( like Movable Type ) or a
python app is 10x more of a hassle

looking at the history of cgi discussions, people often reply with
stuff like “i hate perl” or “proxying to a rails/python app is just
easier”. those aren’t really answers to problems or trying to use
existing software.

not supporting cgi makes sense, but the barrier to using fcgi with
nginx is ‘really fucking high’ right now.

nginx is a great system - i hate that I need to proxy to something
inferior like lighty for ease of administration and setup

// Jonathan V.

w. http://findmeon.com/user/jvanasco
e. r[email protected]

| - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -


| Founder/CEO
| - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -


| FindMeOn.com - The cure for Multiple Web Personality Disorder
| Privacy Minded Web Identity Management and 3D Social Networking
| - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -


setting up fcgi is simple - both in nginx and php. it’s a location
block and a few lines of fastcgi_stuff and you’re good to go.

check out php-fpm for how to make running php fcgi engines a snap.

i’d write more but have to run right now. the wiki has a ton of
examples.

i second mike’s notion. using php-fpm is alot easier to manage fastcgi
php.
I just wish that it would get incorporated into main php release so that
it
would have good fastcgi support out of the box.

On Tue, Sep 23, 2008 at 4:10 PM, Rob S. [email protected]
wrote:

i second mike’s notion. using php-fpm is alot easier to manage fastcgi php.
I just wish that it would get incorporated into main php release so that it
would have good fastcgi support out of the box.

Andrei said he’s planned to get/try to get it into the PHP core when
it is feature complete and he changes the license to be compatible…

I agree. Recompile php with fpm and set php-fpm.conf to listen on
127.0.0.1:9000. Then proxy php to that address and you’re good to go.

There are a couple of “recipes” on the web in addition to the wiki. Look
at http://tajidyakub.com/2008/06/28/nginx-mysql-php-in-centos-51-x86_64/
and http://www.yawn.it/2008/04/30/nginx-php-php-fpm-on-debian-etch-40/ .
Also,
http://articles.slicehost.com/2007/10/19/ubuntu-feisty-adding-an-nginx-init-script#
has a good start up script to use with the start-stop daemon.

On Die 23.09.2008 18:33, Jonathan V. wrote:

I’ve spent the better part of the day combing through docs and the
mailing list trying to get MovableType working with nginx

Please can you tell me ehat do you mean with ‘MovableType’?

it seems the best option is proxying to lighttpd - everything else
seems to involve hacks and copy/pasting scripts off the wiki

Which part of lighty do you use and don’t find in nginx, except the cgi
handling?

For nginx adoption sake, there really has to be a better way.

running php via fcgi is a PITA, and most ways involve using lighttpd’s
fcgi manager trying to get a fcgi compatible perl app ( like Movable
Type ) or a python app is 10x more of a hassle

For perl you can use for example

http://search.cpan.org/~gbjk/FCGI-ProcManager-0.18/ProcManager.pm
FCGI::ProcManager

and for python, there are a lot of fastcgi / http implementation out
there.

But yes, the user must adopt, in the most cases the application.

looking at the history of cgi discussions, people often reply with
stuff like “i hate perl” or “proxying to a rails/python app is just
easier”. those aren’t really answers to problems or trying to use
existing software.

Yes, a cgi support will be great, but anybody must sit down and code it
:wink:

not supporting cgi makes sense, but the barrier to using fcgi with
nginx is ‘really fucking high’ right now.

Depend on the knowledge and the leisure of the user.

If the user can’t ‘speak’ the language in which the script is written,
yes you are right.

As I written before, it would be great to have a cgi interface but I
thinkg it is not so easy to develop a secure high traffic cgi interface.

nginx is a great system - i hate that I need to proxy to something
inferior like lighty for ease of administration and setup

Full ack.

BR

Aleks

On Sep 24, 2008, at 2:55 AM, Aleksandar L. wrote:

Please can you tell me ehat do you mean with ‘MovableType’?

MovableType is a popular line of cms/blog software. It is written
in Perl and FCGI compatible.

Which part of lighty do you use and don’t find in nginx, except the
cgi
handling?

At the moment the only thing I care about is the FCGI integration.
It’s not the ability to use FCGI , its the ease of depolying and
managing apps.

For perl you can use for example

http://search.cpan.org/~gbjk/FCGI-ProcManager-0.18/ProcManager.pm
FCGI::ProcManager

and for python, there are a lot of fastcgi / http implementation out
there.

But yes, the user must adopt, in the most cases the application.

Yes, but my issue isn’t with saying “I need to write a FCGI app” – I
have an FCGI app. I actually have a bunch of them… my admin time
is 10x longer deploying them in an nginx environment right now than a
lighttpd one.

interface.
Yes – depending on the knowledge/leisure of the user.

I’ve been running nginx for 3 years now - having found it relatively
‘early on’ as an alternative to lighttpd leaking memory like a
sieve. The most widely recommended php deployment is still the same
– install lighttpd and use its spawn-fcgi process. it’s neat that
the ‘smart work’ is already done, but given lighttpd’s licensing -
it would be a much easier admin if that script were just provided as
part of nginx.

I have a handful of nginx boxes on freebsd and another group on
ubuntu and RHEL. because nginx puts off so much more of the fcgi
config into other apps , as an admin trying to sync up fcgi support
on the app takes more time because I can’t centralize it to the nginx
install of conf files ( which can easily be handled in version
control and deployed to many servers )

I don’t care about running CGI personally… thought it would be great.

I simply want the integration of running FCGI stuff to be easier
within nginx – and running FCGI apps on nginx to be as easy as
running them under other servers.

// Jonathan V.

w. http://findmeon.com/user/jvanasco
e. [email protected]

| - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -


| Founder/CEO
| - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -


| FindMeOn.com - The cure for Multiple Web Personality Disorder
| Privacy Minded Web Identity Management and 3D Social Networking
| - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -


If you’re running something other than PHP, supervisord (
http://supervisord.org/) is a great language-neutral process manager.
It
can manage any type of FastCGI process.

Unfortunately, the FCGI support hasn’t been released yet but it’s
well-tested in the trunk. http://svn.supervisord.org/

Here’s a sample config

[fcgi-program:foo]
socket=tcp://127.0.0.1:4000
process_name = %(program_name)s_%(process_num)s
command = /path/to/foo.fcgi
numprocs = 5

On Thu, Sep 25, 2008 at 10:07:26AM -0400, Jonathan V. wrote:

Yes, but my issue isn’t with saying “I need to write a FCGI app” – I
have an FCGI app. I actually have a bunch of them… my admin time is
10x longer deploying them in an nginx environment right now than a
lighttpd one.

care to elaborate how deployment of on lighttpd is 10x faster than
deployment
on nginx?

I have a handful of nginx boxes on freebsd and another group on ubuntu
and RHEL. because nginx puts off so much more of the fcgi config into
other apps , as an admin trying to sync up fcgi support on the app takes
more time because I can’t centralize it to the nginx install of conf
files ( which can easily be handled in version control and deployed to
many servers )

care to elaborate why you can’t put the the nginx config directory under
versioning control?

care to elaborate how deployment of on lighttpd is 10x faster than
deployment
on nginx?

At some point as a person who has also used lighty for quite some time I
can
understand the initial posters thoughts (which maybe is not a slowdown
of
deployment rather tricky part of configuring) and here is why:

Basically because nginx lacks global locations (correct me if I am wrong
but
afaik now they are only in server scope) and if() works pretty nasty -
it is
sometimes tricky in case of virtualhosts to make the config look
clean/nice
and work as expected.

Like with the if() case
http://article.gmane.org/gmane.comp.web.nginx.english/6908

In lighty it works the simple way:

some_global_setting = value;

if(something) {
some_global_setting = other_value_A;
}

if(something_else) {
some_global_setting = other_value_B;
}

Also maybe its not the common case but if you have multiple hosts you
have
to specify location ~ .php$ { fastcgi_pass … } for every single
server {}
where the php itself is rarely different for each host. In lighty you
can
specify it globally - fastcgi.server = (".php" => ( host))) … and if
you
need a special configuration/version of php override it in other if().

Basically in lighty you can put everything in if() and keep some
settings
globally (like php backend / expire/gzip settings for certain
file-types/location) but in nginx you can’t… Which makes my nginx
config
at least twice (multiply by virtualhosts) the size of a lighty config
file…

On the other hand don’t see this as whining…
I like nginx and we have solved a lot of problems (like memory leaks and
500
bad gateway errors) by switching to it. Which is primary.
Just would like to see some global location option (so I dont have to
put
location *.jpg expire 30d for every vhost) and maybe make the IF() more
flexible.

rr

I’ve been running nginx for 3 years now - having found it relatively
‘early on’ as an alternative to lighttpd leaking memory like a sieve.
The most widely recommended php deployment is still the same – install
lighttpd and use its spawn-fcgi process. it’s neat that the ‘smart work’
is already done, but given lighttpd’s licensing - it would be a much
easier admin if that script were just provided as part of nginx.

Basically you dont need ‘spawn-fcgi’ from lighty to run php (for
example).
Because php by default has the ability to spawn and control the childs
on
its own…
More or less (besides setuid maybe) all the options are directly
available
from the command line or by setting global env variables. So using a
package
from different software is more like a (free) choice and not necessity.

I myself have sticked to spawnfcgi for couple of years while decided to
switch to php-fpm at last and now I kinda regret a bit I haven’t done it
earlier - much more control (like request_slowlog_timeout and
request_terminate_timeout are godlike), much less trouble…

I simply want the integration of running FCGI stuff to be easier within
nginx – and running FCGI apps on nginx to be as easy as running them
under other servers.

Well how hard is it now?
You provide a host/port on which the FCGI app listens on in nginx and
thats
it… Or I dont get something here?

rr

On Thu, Sep 25, 2008 at 11:41 AM, Reinis R. [email protected] wrote:

I like nginx and we have solved a lot of problems (like memory leaks and 500
bad gateway errors) by switching to it. Which is primary.

me too. every webserver i had would give me a 500 error every so
often. under nginx my load balancer gets an invalid read (not sure
why, can’t see a spike in cpu, memory, anything to give me a hint)
maybe once or twice a week over 3 servers serving up 4-5 million http
requests a day. that’s a pretty good ratio to me.

Just would like to see some global location option (so I dont have to put
location *.jpg expire 30d for every vhost) and maybe make the IF() more
flexible.

i’ve wound up creating expires.conf for my expires location {} and
just include it where i need it. it’s still one line in every server
{} or location {} block…

i agree, it would be nice for some more things to be capable of being
global. the “alias” feature is one that is a bit confusing and doesn’t
work exactly like lighty’s, and we had a pretty simple but effective
global alias defined across the server…

On Don 25.09.2008 10:07, Jonathan V. wrote:

[snipp]

But yes, the user must adopt, in the most cases the application.

Yes, but my issue isn’t with saying “I need to write a FCGI app” – I
have an FCGI app. I actually have a bunch of them… my admin time is
10x longer deploying them in an nginx environment right now than a
lighttpd one.

Ok, thanks, I think I understand you now much better.

I’ve been running nginx for 3 years now - having found it relatively
‘early on’ as an alternative to lighttpd leaking memory like a sieve.
The most widely recommended php deployment is still the same –
install lighttpd and use its spawn-fcgi process. it’s neat that the
‘smart work’ is already done, but given lighttpd’s licensing - it
would be a much easier admin if that script were just provided as part
of nginx.

Have you give the php-fm a chance?

I simply want the integration of running FCGI stuff to be easier
within nginx – and running FCGI apps on nginx to be as easy as
running them under other servers.

Have you some suggestions how/what nginx should change that this is
possible?

Let’s say easier ;-).

What do you wish for nginx to make your live easier, of course to
anybody of this list :wink:

BR

Aleks

On Thu, Sep 25, 2008 at 8:14 AM, Reinis R. [email protected] wrote:

Basically you dont need ‘spawn-fcgi’ from lighty to run php (for example).
Because php by default has the ability to spawn and control the childs on
its own…

exactly! this is all it does…

set env PHP_FCGI_MAX_REQUESTS
set env PHP_FCGI_CHILDREN
php -b $port!

php-fpm is much better, it allows for a nice config, easily set
uid/gid for each group, proper max requests / children - spawn-fcgi
for me did not seem to honor the max requests so my php engines never
restarted… kind of a big feature to be missing.

before php-fpm i just made my own upstart (on ubuntu) script which
basically did this

env PHP_FCGI_MAX_REQUESTS=500
env PHP_FCGI_CHILDREN=10
su -u $user /usr/local/bin/php -b $port

and it recycled the engines properly etc. but it wasn’t as simple as
php-fpm.

On Fri, Sep 26, 2008 at 12:57:48PM +0300, Reinis R. wrote:

  1. Global location ( location / in http scope ) which would affect all
    server {} blocks

Never. I do want to allow unscaleable and unmaintainable configurations.

  1. To be able to use IF() for majority of config options (I suppose this is
    the tricky part) and that it would ‘stack’ rather than use the last valid
    statement.

This will be available when rewrite module will be replace with
some script module.

Never. I do want to allow unscaleable and unmaintainable configurations.

Igor can you a bit explain in what way that turns out unscalable or
unmaintable (are the any internal restrictions to that)?

I mean in most webservers you can specify the ‘handlers’ globaly like
for
‘*.php’ ‘.cgi’ etc etc…
And In most cases (imho) the only thing which changes within a
virtualhost
is the document root. In nginx you have to redefine everything - which
might
be nice as it gives the isolation between seperate vhosts then again is
it
used so often?

Of course we can’t object to your decisions (otherways nginx wouldn’t be
here as it is) but maybe you can give some reasons behind?

rr

On Fri, Sep 26, 2008 at 02:02:50PM +0400, Igor S. wrote:

Never. I do want to allow unscaleable and unmaintainable configurations.

any chance to add ability to use something ala `unscalable on;’?

----- Original Message -----
From: “Aleksandar L.”

What do you wish for nginx to make your live easier, of course to
anybody of this list :wink:

So its allready Christmas time? :slight_smile:

But as I have allready written things I’d like to see in nginx would be:

  1. Global location ( location / in http scope ) which would affect all
    server {} blocks
  2. To be able to use IF() for majority of config options (I suppose this
    is
    the tricky part) and that it would ‘stack’ rather than use the last
    valid
    statement.

rr

Mod +5, Funny

Jonathan V. wrote:

running php via fcgi is a PITA, and most ways involve using lighttpd’s
fcgi manager
Sorry for the late reply… I’ve been away.

PHP via FastCGI is easy. Just execute php-cgi -b /tmp/php.socket &
from the command-line, and its running in FastCGI mode. export PHP_FCGI_CHILDREN=X to set the number of child processes before-hand,
if you need to. I’ve written a small init script to handle this for me.

Then tell nginx to fastcgi_pass to the socket: fastcgi_pass
unix:/tmp/php.socket

I’m running this set-up on 3 production servers with no problem.

Going down the route of using lighty’s spawn-fcgi caused no end of
problems for me. It seems as though the spawn-fcgi binary introduces
some type of race-condition when passing requests to PHP, causing
nginx/lighty to loose connections.

trying to get a fcgi compatible perl app ( like Movable Type ) or a
python app is 10x more of a hassle
Isn’t it just a case of telling the app to listen on a socket and then
pointing nginx to that socket?
not supporting cgi makes sense, but the barrier to using fcgi with
nginx is ‘really fucking high’ right now.
Do we really need the obscenities?

I think mike knocked up a small SCGI module after I put in a request.
Not sure if anyone in the community is testing/using it, but maybe that
could be a starting point for your issues if FastCGI isn’t a solution?

Phillip B Oldham
The Activity People
[email protected] mailto:[email protected]


Policies

This e-mail and its attachments are intended for the above named
recipient(s) only and may be confidential. If they have come to you in
error, please reply to this e-mail and highlight the error. No action
should be taken regarding content, nor must you copy or show them to
anyone.

This e-mail has been created in the knowledge that Internet e-mail is
not a 100% secure communications medium, and we have taken steps to
ensure that this e-mail and attachments are free from any virus. We must
advise that in keeping with good computing practice the recipient should
ensure they are completely virus free, and that you understand and
observe the lack of security when e-mailing us.