# Nginx, fcgi+perl forking a new process per request **URL:** \ **Category:** NGINX\ **Created:** [March 15, 2010, 1:13pm UTC](https://www.ruby-forum.com/t/nginx-fcgi-perl-forking-a-new-process-per-request/185164 "2010-03-15T13:13:55Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![iamstef](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/6a8cbe/32.png) [@iamstef](https://www.ruby-forum.com/u/iamstef)\ **Post date:** [March 15, 2010, 1:13pm UTC](https://www.ruby-forum.com/t/nginx-fcgi-perl-forking-a-new-process-per-request/185164/1 "2010-03-15T13:13:55Z")
Hi, Im trying to experiment more regarding nginx + fcgi,perl. Im using the following configuration: OS: Solaris 10 x86 Perl CGI support: nginx-fcgi-0.4.3 > **[nginx.eu](http://www.nginx.eu/nginx-fcgi.html)** > > This domain may be for sale! worker\_processes 1; ``` location ~ ^/cgi-bin/.*\.cgi$ { gzip off; fastcgi_pass ``` unix:/opt/sdr/report/ws/fastcgi\_temp/nginx-fcgi.sock; fastcgi\_read\_timeout 5m; fastcgi\_index index.cgi; # # You may copy and paste the lines under or use include directive # include /etc/nginx/nginx-fcgi.conf; # In this example all is in one file # fastcgi\_param SCRIPT\_FILENAME /opt/sdr/report/docroot/$fastcgi\_script\_name; fastcgi\_param QUERY\_STRING $query\_string; fastcgi\_param REQUEST\_METHOD $request\_method; fastcgi\_param CONTENT\_TYPE $content\_type; fastcgi\_param CONTENT\_LENGTH $content\_length; fastcgi\_param GATEWAY\_INTERFACE CGI/1.1; fastcgi\_param SERVER\_SOFTWARE nginx; fastcgi\_param SCRIPT\_NAME $fastcgi\_script\_name; fastcgi\_param REQUEST\_URI $request\_uri; fastcgi\_param DOCUMENT\_URI $document\_uri; fastcgi\_param DOCUMENT\_ROOT $document\_root; fastcgi\_param SERVER\_PROTOCOL $server\_protocol; fastcgi\_param REMOTE\_ADDR $remote\_addr; fastcgi\_param REMOTE\_PORT $remote\_port; fastcgi\_param SERVER\_ADDR $server\_addr; fastcgi\_param SERVER\_PORT $server\_port; fastcgi\_param SERVER\_NAME $server\_name; } I have all processes running, ptree command: 5548 /opt/sdr/report/perl/bin/perl /opt/sdr/report/ws/sbin/nginx-fcgi -l /opt/sdr/re 5552 /opt/sdr/report/ws/sbin/nginx -c /opt/sdr/report/ws/conf/nginx.conf 5553 /opt/sdr/report/ws/sbin/nginx -c /opt/sdr/report/ws/conf/nginx.conf When I kick a jmeter test against the server I can see dozens of processes forked by the 5548 the master FCGI process. As understood I should not see this behavior when using FastCGI, isnt it ? Example, using DTrace: # /opt/csw/bin/execsnoop UID PID PPID ARGS 60001 18035 5548 /opt/sdr/report/perl/bin/perl -wT /opt/sdr/report/docroot//cgi-bin/initial.cgi 60001 18038 5548 /opt/sdr/report/perl/bin/perl -wT /opt/sdr/report/docroot//cgi-bin/initial.cgi 60001 18039 5548 /opt/sdr/report/perl/bin/perl -wT /opt/sdr/report/docroot//cgi-bin/initial.cgi 60001 18040 5548 /opt/sdr/report/perl/bin/perl -wT /opt/sdr/report/docroot//cgi-bin/initial.cgi 60001 18041 5548 /opt/sdr/report/perl/bin/perl -wT /opt/sdr/report/docroot//cgi-bin/initial.cgi 60001 18042 5548 /opt/sdr/report/perl/bin/perl -wT /opt/sdr/report/docroot//cgi-bin/initial.cgi 60001 18044 5548 /opt/sdr/report/perl/bin/perl -wT /opt/sdr/report/docroot//cgi-bin/initial.cgi 60001 18045 5548 /opt/sdr/report/perl/bin/perl -wT /opt/sdr/report/docroot//cgi-bin/initial.cgi 60001 18046 5548 /opt/sdr/report/perl/bin/perl -wT /opt/sdr/report/docroot//cgi-bin/initial.cgi 60001 18031 5548 /opt/sdr/report/perl/bin/perl -wT /opt/sdr/report/docroot//cgi-bin/initial.cgi 60001 18032 5548 /opt/sdr/report/perl/bin/perl -wT /opt/sdr/report/docroot//cgi-bin/initial.cgi 60001 18033 5548 /opt/sdr/report/perl/bin/perl -wT /opt/sdr/report/docroot//cgi-bin/initial.cgi 60001 18034 5548 /opt/sdr/report/perl/bin/perl -wT /opt/sdr/report/docroot//cgi-bin/initial.cgi 60001 18036 5548 /opt/sdr/report/perl/bin/perl -wT /opt/sdr/report/docroot//cgi-bin/initial.cgi 60001 18037 5548 /opt/sdr/report/perl/bin/perl -wT /opt/sdr/report/docroot//cgi-bin/initial.cgi 60001 18043 5548 /opt/sdr/report/perl/bin/perl -wT /opt/sdr/report/docroot//cgi-bin/initial.cgi 60001 18047 5548 /opt/sdr/report/perl/bin/perl -wT /opt/sdr/report/docroot//cgi-bin/initial.cgi 60001 18048 5548 /opt/sdr/report/perl/bin/perl -wT /opt/sdr/report/docroot//cgi-bin/initial.cgi […] Any ideas ? Thanks, Stefan ---
**Author:** ![iamstef](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/6a8cbe/32.png) [@iamstef](https://www.ruby-forum.com/u/iamstef)\ **Post date:** [March 15, 2010, 1:27pm UTC](https://www.ruby-forum.com/t/nginx-fcgi-perl-forking-a-new-process-per-request/185164/2 "2010-03-15T13:27:23Z")
Hello! On Mon, Mar 15, 2010 at 02:13:28PM +0200, Stefan P. wrote: > Im trying to experiment more regarding nginx + fcgi,perl. Im using the > following configuration: > > OS: Solaris 10 x86 > Perl CGI support: nginx-fcgi-0.4.3 > [http://www.nginx.eu/nginx-fcgi.html](http://www.nginx.eu/nginx-fcgi.html) […] > I have all processes running, ptree command: > > 5548 /opt/sdr/report/perl/bin/perl /opt/sdr/report/ws/sbin/nginx-fcgi -l /opt/sdr/re > 5552 /opt/sdr/report/ws/sbin/nginx -c /opt/sdr/report/ws/conf/nginx.conf > 5553 /opt/sdr/report/ws/sbin/nginx -c /opt/sdr/report/ws/conf/nginx.conf > > When I kick a jmeter test against the server I can see dozens of processes > forked by the 5548 the master FCGI process. As understood I should not > see this behavior when using FastCGI, isnt it ? 1. FastCGI is just protocol, nothing more. Implementation details (including number of forked processes) are up to FastCGI application. 2. Script you refer to is just wrapper which executes CGI scripts. And it does so by running separate process for each request. Maxim D. ---
**Author:** ![iamstef](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/6a8cbe/32.png) [@iamstef](https://www.ruby-forum.com/u/iamstef)\ **Post date:** [March 15, 2010, 5:24pm UTC](https://www.ruby-forum.com/t/nginx-fcgi-perl-forking-a-new-process-per-request/185164/3 "2010-03-15T17:24:29Z")
On Mon, Mar 15, 2010 at 12:43, Stefan P. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > In my case nginx-fcgi does currently the job and it is not > a very solid solution. Correct ? A single page load can - and most often does - have more than one request; a webserver gets hits by more than oner person… The higher the traffic the higher the chances of you fork-bombing your own server. It might reach the maximum allowed number of processes, if defined, or whichever maximum size for the PID variable (will-guessing this one)? Nice academic exercise. Anyway, you should look for a fastcgi daemon instead of a script; anything that won’t spwan 1 process per 1 request. On Debian i use the php-cgi package and a fastcgi script to spawn the daemon on /etc/init.d. It uses 4-6 processes top, it’s attached. HTH Nuno ---
**Author:** ![iamstef](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/6a8cbe/32.png) [@iamstef](https://www.ruby-forum.com/u/iamstef)\ **Post date:** [March 15, 2010, 5:39pm UTC](https://www.ruby-forum.com/t/nginx-fcgi-perl-forking-a-new-process-per-request/185164/4 "2010-03-15T17:39:43Z")
> A single page load can - and most often does - have more than one > request; a webserver gets hits by more than oner person… The higher > the traffic the higher the chances of you fork-bombing your own > server. It might reach the maximum allowed number of processes, if > defined, or whichever maximum size for the PID variable (will-guessing > this one)? Nice academic exercise. sure, makes sense. > Anyway, you should look for a fastcgi daemon instead of a script; > anything that won’t spwan 1 process per 1 request. On Debian i use the > php-cgi package and a fastcgi script to spawn the daemon on > /etc/init.d. It uses 4-6 processes top, it’s attached. Thanks. Probable a fastcgi daemon written in C or Perl. Found out the Catalyst::Engine::FastCGI , FCGI::Engine. I would stick with the current version nginx-fcgi for time being. It would be nice to have a std fastcgi daemon written in C, part of NGINX. But most likely this is not the purpose of the project. If you guys have any ideas about a simple C based fastcgi daemon let me know. Would be cool if would compile across Solaris, Linux, FreeBSD. Cheers, Stefan ---
**Author:** ![iamstef](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/6a8cbe/32.png) [@iamstef](https://www.ruby-forum.com/u/iamstef)\ **Post date:** [March 15, 2010, 1:43pm UTC](https://www.ruby-forum.com/t/nginx-fcgi-perl-forking-a-new-process-per-request/185164/5 "2010-03-15T13:43:47Z")
Hi Maxim, > > When I kick a jmeter test against the server I can see dozens of processes > > forked by the 5548 the master FCGI process. As understood I should not > > see this behavior when using FastCGI, isnt it ? > > 1. FastCGI is just protocol, nothing more. Implementation details > (including number of forked processes) are up to FastCGI > application. In this case, the nginx-fcgi daemon, which is a perl process, handling the CGI things. > 1. Script you refer to is just wrapper which executes CGI scripts. > And it does so by running separate process for each request. Hmmm. Right. So basically if we want to have a solid FCGI daemon, one can write our own FCGI app which will receive the requests from nginx. Right ? In my case nginx-fcgi does currently the job and it is not a very solid solution. Correct ? stefan ---
**Author:** ![iamstef](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/6a8cbe/32.png) [@iamstef](https://www.ruby-forum.com/u/iamstef)\ **Post date:** [March 15, 2010, 5:50pm UTC](https://www.ruby-forum.com/t/nginx-fcgi-perl-forking-a-new-process-per-request/185164/6 "2010-03-15T17:50:15Z")
There is a standard C library for FCGI. [http://www.fastcgi.com/drupal/node/5](http://www.fastcgi.com/drupal/node/5) On Mon, Mar 15, 2010 at 9:38 AM, Stefan P. \ **Author:** ![iamstef](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/6a8cbe/32.png) [@iamstef](https://www.ruby-forum.com/u/iamstef)\ **Post date:** [March 15, 2010, 9:16pm UTC](https://www.ruby-forum.com/t/nginx-fcgi-perl-forking-a-new-process-per-request/185164/7 "2010-03-15T21:16:23Z") On pon, mar 15, 2010 at 06:38:54 +0200, Stefan P. wrote: > It would be nice to have a std fastcgi daemon written in C, > part of NGINX. But most likely this is not the purpose of > the project. > > If you guys have any ideas about a simple C based fastcgi daemon > let me know. Would be cool if would compile across Solaris, Linux, > FreeBSD. > **[GitHub - gnosek/fcgiwrap: Simple FastCGI wrapper for CGI scripts](https://github.com/gnosek/fcgiwrap)** > > Simple FastCGI wrapper for CGI scripts [http://nginx.localdomain.pl/wiki/FcgiWrap](http://nginx.localdomain.pl/wiki/FcgiWrap) Shout if it doesn’t compile on your OS. Needs libfcgi (with headers) installed. Best regards, Grzegorz N. ---
**Author:** ![iamstef](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/6a8cbe/32.png) [@iamstef](https://www.ruby-forum.com/u/iamstef)\ **Post date:** [March 15, 2010, 8:33pm UTC](https://www.ruby-forum.com/t/nginx-fcgi-perl-forking-a-new-process-per-request/185164/8 "2010-03-15T20:33:02Z")
On 09:49 Mon 15 Mar , Roger H. wrote: > There is a standard C library for FCGI. > [http://www.fastcgi.com/drupal/node/5](http://www.fastcgi.com/drupal/node/5) thanks. I know. I read a bit the std and looked over. Im not fluent in C but if I will have time I will think something - probable not very trivial to write the FCGI server in C. Stefan ---
**Author:** ![iamstef](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/6a8cbe/32.png) [@iamstef](https://www.ruby-forum.com/u/iamstef)\ **Post date:** [March 16, 2010, 10:56am UTC](https://www.ruby-forum.com/t/nginx-fcgi-perl-forking-a-new-process-per-request/185164/9 "2010-03-16T10:56:49Z")
On Monday of March 15 2010, Stefan P. wrote: > Found out the FCGI::Engine I use FCGI::Engine with nginx (on FreeBSD) and it works great. It’s pretty busy internal application (around 30 fcgi child processes) migrated from mod\_perl2 because of memory problems/leaks. ## regards Marcin G., PGP 0x9F183FA3 jabber jid:removed\_email\_address@domain.invalid, gg:2532994 [http://the.fork.pl](http://the.fork.pl) ---
**Author:** ![iamstef](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/6a8cbe/32.png) [@iamstef](https://www.ruby-forum.com/u/iamstef)\ **Post date:** [March 20, 2010, 1:37pm UTC](https://www.ruby-forum.com/t/nginx-fcgi-perl-forking-a-new-process-per-request/185164/10 "2010-03-20T13:37:56Z")
Here is a short review about a simple FCGI comparation: perl/native on Solaris 10U8, using nginx 32bit. Thanks to gnosek for pointers about the native version. [http://systemdatarecorder.org:9009/bugzilla/show\_bug.cgi?id=51](http://systemdatarecorder.org:9009/bugzilla/show_bug.cgi?id=51) stefan ---
**Author:** ![iamstef](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/6a8cbe/32.png) [@iamstef](https://www.ruby-forum.com/u/iamstef)\ **Post date:** [March 15, 2010, 10:08pm UTC](https://www.ruby-forum.com/t/nginx-fcgi-perl-forking-a-new-process-per-request/185164/11 "2010-03-15T22:08:28Z")
> [GitHub - gnosek/fcgiwrap: Simple FastCGI wrapper for CGI scripts · GitHub](http://github.com/gnosek/fcgiwrap) > [http://nginx.localdomain.pl/wiki/FcgiWrap](http://nginx.localdomain.pl/wiki/FcgiWrap) > > Shout if it doesn’t compile on your OS. Needs libfcgi (with headers) > installed. Huh, thats very nice. Thanks a lot. I will post later the results. Stefan ---
**Author:** ![iamstef](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/6a8cbe/32.png) [@iamstef](https://www.ruby-forum.com/u/iamstef)\ **Post date:** [March 21, 2010, 1:09pm UTC](https://www.ruby-forum.com/t/nginx-fcgi-perl-forking-a-new-process-per-request/185164/12 "2010-03-21T13:09:10Z")
> Interesting. As for the large response time, you’d generally run a bunch > of fcgiwraps (e.g. by using the -c option) to improve responsiveness. > The memory cost should be negligible compared to a single instance. Cheers. Yes indeed it makes a difference. ####################################### Test 4: N=200, Z=5sec, T=10min Outputs: X=35.6/sec, R=118ms, Err=0.00% ####################################### so from 1872 to 118ms using 3 FCGI processes. > Also, if your real CGI scripts you are going to run are all Perl-based, > you may get better results by converting them to talk FastCGI directly > to save on the overhead of launching Perl and loading its modules once > per request. However, this includes some gotchas and is generally > getting way off topic 🙂 I will document all of these for future. SDR Reporting is and will be only CGI Perl so I will deploy fcgiwraps as part of the final solution. As for improving these numbers I will open a CR in bugzilla to keep track of all of these. thanks for help, Stefan ---
**Author:** ![iamstef](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/6a8cbe/32.png) [@iamstef](https://www.ruby-forum.com/u/iamstef)\ **Post date:** [March 20, 2010, 7:15pm UTC](https://www.ruby-forum.com/t/nginx-fcgi-perl-forking-a-new-process-per-request/185164/13 "2010-03-20T19:15:05Z")
On sob, mar 20, 2010 at 02:37:07 +0200, Stefan P. wrote: > Here is a short review about a simple FCGI comparation: perl/native > on Solaris 10U8, using nginx 32bit. > > Thanks to gnosek for pointers about the native version. > > [http://systemdatarecorder.org:9009/bugzilla/show\_bug.cgi?id=51](http://systemdatarecorder.org:9009/bugzilla/show_bug.cgi?id=51) Interesting. As for the large response time, you’d generally run a bunch of fcgiwraps (e.g. by using the -c option) to improve responsiveness. The memory cost should be negligible compared to a single instance. Right now all the requests are processed sequentially, so if your CGI script slept for 5 seconds, you’d get 0.2 req/sec tops plus an absolutely abysmal response time. Of course, this also applies to the Perl wrapper. You can implement a simplistic prefork there by replacing: * * * my $pid = fork(); ## if( $pid == 0 ) { &main; exit 0; } with: * * * for (1 .. HOWEVER\_MANY\_CHILDREN\_YOU\_WANT) { my $pid = fork(); ## if( $pid == 0 ) { &main; exit 0; } } Also, if your real CGI scripts you are going to run are all Perl-based, you may get better results by converting them to talk FastCGI directly to save on the overhead of launching Perl and loading its modules once per request. However, this includes some gotchas and is generally getting way off topic 🙂 Best regards, Grzegorz N.