Nginx, fcgi+perl forking a new process per request

Server NGINX
#1

Hi,

Im trying to experiment more regarding nginx + fcgi,perl. Im using the
following configuration:

OS: Solaris 10 x86
Perl CGI support: nginx-fcgi-0.4.3
http://www.nginx.eu/nginx-fcgi.html

worker_processes 1;

    location ~ ^/cgi-bin/.*\.cgi$ {
        gzip off;
        fastcgi_pass

unix:/opt/sdr/report/ws/fastcgi_temp/nginx-fcgi.sock;
fastcgi_read_timeout 5m;
fastcgi_index index.cgi;
#
# You may copy and paste the lines under or use include
directive
# include /etc/nginx/nginx-fcgi.conf;
# In this example all is in one file
#
fastcgi_param SCRIPT_FILENAME
/opt/sdr/report/docroot/$fastcgi_script_name;
fastcgi_param QUERY_STRING $query_string;
fastcgi_param REQUEST_METHOD $request_method;
fastcgi_param CONTENT_TYPE $content_type;
fastcgi_param CONTENT_LENGTH $content_length;
fastcgi_param GATEWAY_INTERFACE CGI/1.1;
fastcgi_param SERVER_SOFTWARE nginx;
fastcgi_param SCRIPT_NAME $fastcgi_script_name;
fastcgi_param REQUEST_URI $request_uri;
fastcgi_param DOCUMENT_URI $document_uri;
fastcgi_param DOCUMENT_ROOT $document_root;
fastcgi_param SERVER_PROTOCOL $server_protocol;
fastcgi_param REMOTE_ADDR $remote_addr;
fastcgi_param REMOTE_PORT $remote_port;
fastcgi_param SERVER_ADDR $server_addr;
fastcgi_param SERVER_PORT $server_port;
fastcgi_param SERVER_NAME $server_name;
}

I have all processes running, ptree command:

5548 /opt/sdr/report/perl/bin/perl /opt/sdr/report/ws/sbin/nginx-fcgi
-l /opt/sdr/re
5552 /opt/sdr/report/ws/sbin/nginx -c
/opt/sdr/report/ws/conf/nginx.conf
5553 /opt/sdr/report/ws/sbin/nginx -c
/opt/sdr/report/ws/conf/nginx.conf

When I kick a jmeter test against the server I can see dozens of
processes
forked by the 5548 the master FCGI process. As understood I should not
see this behavior when using FastCGI, isnt it ?

Example, using DTrace:

/opt/csw/bin/execsnoop

UID PID PPID ARGS
60001 18035 5548 /opt/sdr/report/perl/bin/perl -wT
/opt/sdr/report/docroot//cgi-bin/initial.cgi
60001 18038 5548 /opt/sdr/report/perl/bin/perl -wT
/opt/sdr/report/docroot//cgi-bin/initial.cgi
60001 18039 5548 /opt/sdr/report/perl/bin/perl -wT
/opt/sdr/report/docroot//cgi-bin/initial.cgi
60001 18040 5548 /opt/sdr/report/perl/bin/perl -wT
/opt/sdr/report/docroot//cgi-bin/initial.cgi
60001 18041 5548 /opt/sdr/report/perl/bin/perl -wT
/opt/sdr/report/docroot//cgi-bin/initial.cgi
60001 18042 5548 /opt/sdr/report/perl/bin/perl -wT
/opt/sdr/report/docroot//cgi-bin/initial.cgi
60001 18044 5548 /opt/sdr/report/perl/bin/perl -wT
/opt/sdr/report/docroot//cgi-bin/initial.cgi
60001 18045 5548 /opt/sdr/report/perl/bin/perl -wT
/opt/sdr/report/docroot//cgi-bin/initial.cgi
60001 18046 5548 /opt/sdr/report/perl/bin/perl -wT
/opt/sdr/report/docroot//cgi-bin/initial.cgi
60001 18031 5548 /opt/sdr/report/perl/bin/perl -wT
/opt/sdr/report/docroot//cgi-bin/initial.cgi
60001 18032 5548 /opt/sdr/report/perl/bin/perl -wT
/opt/sdr/report/docroot//cgi-bin/initial.cgi
60001 18033 5548 /opt/sdr/report/perl/bin/perl -wT
/opt/sdr/report/docroot//cgi-bin/initial.cgi
60001 18034 5548 /opt/sdr/report/perl/bin/perl -wT
/opt/sdr/report/docroot//cgi-bin/initial.cgi
60001 18036 5548 /opt/sdr/report/perl/bin/perl -wT
/opt/sdr/report/docroot//cgi-bin/initial.cgi
60001 18037 5548 /opt/sdr/report/perl/bin/perl -wT
/opt/sdr/report/docroot//cgi-bin/initial.cgi
60001 18043 5548 /opt/sdr/report/perl/bin/perl -wT
/opt/sdr/report/docroot//cgi-bin/initial.cgi
60001 18047 5548 /opt/sdr/report/perl/bin/perl -wT
/opt/sdr/report/docroot//cgi-bin/initial.cgi
60001 18048 5548 /opt/sdr/report/perl/bin/perl -wT
/opt/sdr/report/docroot//cgi-bin/initial.cgi

[…]

Any ideas ?

Thanks,
Stefan

#2

Hello!

On Mon, Mar 15, 2010 at 02:13:28PM +0200, Stefan P. wrote:

Im trying to experiment more regarding nginx + fcgi,perl. Im using the
following configuration:

OS: Solaris 10 x86
Perl CGI support: nginx-fcgi-0.4.3
http://www.nginx.eu/nginx-fcgi.html

[…]

I have all processes running, ptree command:

5548 /opt/sdr/report/perl/bin/perl /opt/sdr/report/ws/sbin/nginx-fcgi -l /opt/sdr/re
5552 /opt/sdr/report/ws/sbin/nginx -c /opt/sdr/report/ws/conf/nginx.conf
5553 /opt/sdr/report/ws/sbin/nginx -c /opt/sdr/report/ws/conf/nginx.conf

When I kick a jmeter test against the server I can see dozens of processes
forked by the 5548 the master FCGI process. As understood I should not
see this behavior when using FastCGI, isnt it ?

  1. FastCGI is just protocol, nothing more. Implementation details
    (including number of forked processes) are up to FastCGI
    application.

  2. Script you refer to is just wrapper which executes CGI scripts.
    And it does so by running separate process for each request.

Maxim D.

#3

On Mon, Mar 15, 2010 at 12:43, Stefan P.
[email protected] wrote:

In my case nginx-fcgi does currently the job and it is not
a very solid solution. Correct ?

A single page load can - and most often does - have more than one
request; a webserver gets hits by more than oner person… The higher
the traffic the higher the chances of you fork-bombing your own
server. It might reach the maximum allowed number of processes, if
defined, or whichever maximum size for the PID variable (will-guessing
this one)? Nice academic exercise.

Anyway, you should look for a fastcgi daemon instead of a script;
anything that won’t spwan 1 process per 1 request. On Debian i use the
php-cgi package and a fastcgi script to spawn the daemon on
/etc/init.d. It uses 4-6 processes top, it’s attached.

HTH
Nuno

#4

A single page load can - and most often does - have more than one
request; a webserver gets hits by more than oner person… The higher
the traffic the higher the chances of you fork-bombing your own
server. It might reach the maximum allowed number of processes, if
defined, or whichever maximum size for the PID variable (will-guessing
this one)? Nice academic exercise.

sure, makes sense.

Anyway, you should look for a fastcgi daemon instead of a script;
anything that won’t spwan 1 process per 1 request. On Debian i use the
php-cgi package and a fastcgi script to spawn the daemon on
/etc/init.d. It uses 4-6 processes top, it’s attached.

Thanks. Probable a fastcgi daemon written in C or Perl. Found
out the Catalyst::Engine::FastCGI , FCGI::Engine. I would
stick with the current version nginx-fcgi for time being.

It would be nice to have a std fastcgi daemon written in C,
part of NGINX. But most likely this is not the purpose of
the project.

If you guys have any ideas about a simple C based fastcgi daemon
let me know. Would be cool if would compile across Solaris, Linux,
FreeBSD.

Cheers,
Stefan

#5

Hi Maxim,

When I kick a jmeter test against the server I can see dozens of processes
forked by the 5548 the master FCGI process. As understood I should not
see this behavior when using FastCGI, isnt it ?

  1. FastCGI is just protocol, nothing more. Implementation details
    (including number of forked processes) are up to FastCGI
    application.

In this case, the nginx-fcgi daemon, which is a perl
process, handling the CGI things.

  1. Script you refer to is just wrapper which executes CGI scripts.
    And it does so by running separate process for each request.

Hmmm. Right. So basically if we want to have a solid
FCGI daemon, one can write our own FCGI app which
will receive the requests from nginx. Right ?

In my case nginx-fcgi does currently the job and it is not
a very solid solution. Correct ?

stefan

#6

There is a standard C library for FCGI.

http://www.fastcgi.com/drupal/node/5

On Mon, Mar 15, 2010 at 9:38 AM, Stefan P.
<[email protected]

#7

On pon, mar 15, 2010 at 06:38:54 +0200, Stefan P. wrote:

It would be nice to have a std fastcgi daemon written in C,
part of NGINX. But most likely this is not the purpose of
the project.

If you guys have any ideas about a simple C based fastcgi daemon
let me know. Would be cool if would compile across Solaris, Linux,
FreeBSD.

http://github.com/gnosek/fcgiwrap
http://nginx.localdomain.pl/wiki/FcgiWrap

Shout if it doesn’t compile on your OS. Needs libfcgi (with headers)
installed.

Best regards,
Grzegorz N.

#8

On 09:49 Mon 15 Mar , Roger H. wrote:

There is a standard C library for FCGI.
http://www.fastcgi.com/drupal/node/5

thanks. I know. I read a bit the std and looked
over. Im not fluent in C but if I will have time
I will think something - probable not very trivial
to write the FCGI server in C.

Stefan

#9

On Monday of March 15 2010, Stefan P. wrote:

Found out the FCGI::Engine

I use FCGI::Engine with nginx (on FreeBSD) and it works great. It’s
pretty
busy internal application (around 30 fcgi child processes) migrated from
mod_perl2 because of memory problems/leaks.

regards

Marcin G., PGP 0x9F183FA3
jabber jid:[email protected], gg:2532994
http://the.fork.pl

#10

Here is a short review about a simple FCGI comparation: perl/native
on Solaris 10U8, using nginx 32bit.

Thanks to gnosek for pointers about the native version.

http://systemdatarecorder.org:9009/bugzilla/show_bug.cgi?id=51

stefan

#11

http://github.com/gnosek/fcgiwrap
http://nginx.localdomain.pl/wiki/FcgiWrap

Shout if it doesn’t compile on your OS. Needs libfcgi (with headers)
installed.

Huh, thats very nice. Thanks a lot.
I will post later the results.

Stefan

#12

Interesting. As for the large response time, you’d generally run a bunch
of fcgiwraps (e.g. by using the -c option) to improve responsiveness.
The memory cost should be negligible compared to a single instance.

Cheers. Yes indeed it makes a difference.

#######################################
Test 4: N=200, Z=5sec, T=10min
Outputs: X=35.6/sec, R=118ms, Err=0.00%
#######################################

so from 1872 to 118ms using 3 FCGI processes.

Also, if your real CGI scripts you are going to run are all Perl-based,
you may get better results by converting them to talk FastCGI directly
to save on the overhead of launching Perl and loading its modules once
per request. However, this includes some gotchas and is generally
getting way off topic :slight_smile:

I will document all of these for future. SDR Reporting is
and will be only CGI Perl so I will deploy fcgiwraps as part
of the final solution. As for improving these numbers I will
open a CR in bugzilla to keep track of all of these.

thanks for help,
Stefan

#13

On sob, mar 20, 2010 at 02:37:07 +0200, Stefan P. wrote:

Here is a short review about a simple FCGI comparation: perl/native
on Solaris 10U8, using nginx 32bit.

Thanks to gnosek for pointers about the native version.

http://systemdatarecorder.org:9009/bugzilla/show_bug.cgi?id=51

Interesting. As for the large response time, you’d generally run a bunch
of fcgiwraps (e.g. by using the -c option) to improve responsiveness.
The memory cost should be negligible compared to a single instance.

Right now all the requests are processed sequentially, so if your CGI
script slept for 5 seconds, you’d get 0.2 req/sec tops plus an
absolutely abysmal response time.

Of course, this also applies to the Perl wrapper. You can implement a
simplistic prefork there by replacing:

my $pid = fork();

if( $pid == 0 ) {
&main;
exit 0;
}

with:

for (1 … HOWEVER_MANY_CHILDREN_YOU_WANT) {
my $pid = fork();

if( $pid == 0 ) {
&main;
exit 0;
}
}

Also, if your real CGI scripts you are going to run are all Perl-based,
you may get better results by converting them to talk FastCGI directly
to save on the overhead of launching Perl and loading its modules once
per request. However, this includes some gotchas and is generally
getting way off topic :slight_smile:

Best regards,
Grzegorz N.