Optimizing worker_processes, worker_connections & PHP_FCGI_CHILDREN - Any Good Tutorial?

rahul286 · June 17, 2010, 6:28pm

On 15.06.2010 22:27, Rahul B. wrote:

worker_process = num of CPU. nothing more
I will make this a law for myself.

I recommend have few more workers than number of cores. On one of my
overloaded dual core server I changed 2 worksers to 6, this reduced a
lot IOwait and fixed latency problem when using site which is on this
server. Play a bit with it to found best for you number of workers.

– Piotr.

rahul286 · June 17, 2010, 6:28pm

interesting. Does nginx handle this fcgi response ? You can open a bug
on bugs.php.net to request this feature and, if you have so, propose a
corresponding patch.

It has been requested couple of times:

http://bugs.php.net/bug.php?id=39809
http://bugs.php.net/bug.php?id=43610

… then the response (at that moment) that the php master process can’t
do
that.

I wonder now with the use of libevent in the manager if thats still the
case.

rr

rahul286 · June 17, 2010, 8:36pm

On 15.06.2010 22:48, Rahul B. wrote:

@jabberuser

I recommend have few more workers than number of cores. On one of my
overloaded dual core server I changed 2 worksers to 6, this reduced a
lot IOwait and fixed latency problem when using site which is on this
server. Play a bit with it to found best for you number of workers.

Thanks for suggestions.
I may go you way just in case fastcgi_cache in nginx improves my
performance.
In that case I will reduce value of PHP_FCGI_CHILDREN to offset it.

I have there 6 nginx workers and 10 php childrens, restarted after 100
request each
PHP_FCGI_CHILDREN=10
PHP_FCGI_MAX_REQUESTS=100

Php use ~22M of ram
16.5 MiB + 5.4 MiB = 21.9 MiB php-cgi (11)
And nginx ~5M
2.9 MiB + 1.4 MiB = 4.4 MiB nginx (7)
(base on ps_mem.py script).

nginx workers is one thing and php another. You host only php or also
static files?

– Piotr.

rahul286 · June 17, 2010, 8:36pm

@jerome

no i mean memory used by the kernel to cache local files in order not
to access file on hardrive. see output of free, the column “cached”

I already have that Cache it seems. Following is my output of Free
command.

root@rtblogs:~# free
total used free shared buffers
cached
Mem: 1451740 483524 968216 0 61192
283196
-/+ buffers/cache: 139136 1312604
Swap: 262136 0 262136

I must thanks linode for this. But this always reduced my free memory by
huge amount.
I guess such situations force us to spend more time in deciding on what
to
cache and how much to cache.
We have too many levels where we can cache and good thing about them is
that
they all happily live together.

@jabberuser

I recommend have few more workers than number of cores. On one of my
overloaded dual core server I changed 2 worksers to 6, this reduced a
lot IOwait and fixed latency problem when using site which is on this
server. Play a bit with it to found best for you number of workers.

Thanks for suggestions.
I may go you way just in case fastcgi_cache in nginx improves my
performance.
In that case I will reduce value of PHP_FCGI_CHILDREN to offset it.

-Rahul

rahul286 · June 17, 2010, 8:36pm

Piotr K. Wrote:

Php use ~22M of ram
16.5 MiB + 5.4 MiB = 21.9 MiB php-cgi (11)
And nginx ~5M
2.9 MiB + 1.4 MiB = 4.4 MiB nginx (7)
(base on ps_mem.py script).

I ran ps_mem.py [http://www.pixelbeat.org/scripts/ps_mem.py]
And found…
5.5 MiB + 992.5 KiB = 6.4 MiB nginx (9)
49.1 MiB + 60.7 MiB = 109.7 MiB php-cgi (6)

I guess for PHP, shared 60.7MB contains APC cache of 60MB.
And also total memory used by php-cgi is not 6109.7 but 649.1 + 60

Also my php-cgi is double in size compared to your php-cgi
Should I think of removing php extensions in this case!

Posted at Nginx Forum:

rahul286 · June 17, 2010, 8:36pm

On 15.06.2010 23:28, rahul286 wrote:

5.5 MiB + 992.5 KiB = 6.4 MiB nginx (9)
49.1 MiB + 60.7 MiB = 109.7 MiB php-cgi (6)

I guess for PHP, shared 60.7MB contains APC cache of 60MB.
And also total memory used by php-cgi is not 6109.7 but 649.1 + 60

Also my php-cgi is double in size compared to your php-cgi
Should I think of removing php extensions in this case!

I don’t know why my php use less memory than yours, I am not a php guy
but my server is powered by Funtoo Linux (Gentoo fork) and php is
compiled only with what I need, nothing more.

– Piotr.

rahul286 · June 17, 2010, 8:36pm

Piotr K. Wrote:

nginx workers is one thing and php another. You
host only php or also static files?

I agree with you completely.
I am running a wordpress(es) only as PHP application.

All wordpress static content like images, js/css files are served via
CDN.
So as far as typical static content is concerned nginx have no
considerable load to worry about.

Then another “static” content comes from output of PHP cached as static
html files.
During my last test, after “caching” was done, I stopped fastcgi-php &
mysql server and as expected request still served by server. Of course
back-end was completely down in that case.

It is backend and some portion of front-end which amounts to PHP load.
And I am putting efforts to make backend faster compared to its current
load time.

Posted at Nginx Forum:

rahul286 · June 17, 2010, 8:36pm

@jerome

I don’t know if it’s possible yet, I have to check it out and think
about it. Since FPM has a different design than the php-f?cgi, it’s
time to make a new request on bugs.php.net for FPM (not for php-cgi)

I am using PHP version 5.2.10-2ubuntu6.4
Which is better in this case?
php-FPM or php-fcgi?

Thanks,
Rahul

rahul286 · June 17, 2010, 8:36pm

2010/6/15 Reinis R. [email protected]:

that.

I wonder now with the use of libevent in the manager if thats still the
case.

I don’t know if it’s possible yet, I have to check it out and think
about it. Since FPM has a different design than the php-f?cgi, it’s
time to make a new request on bugs.php.net for FPM (not for php-cgi)

rahul286 · June 18, 2010, 4:39am

Reinis R. Wrote:

there is no need to recompile script each time a
new child is spawned). Not
to mention other benefits like persistant
connections (to memcache / db /
etc) which aren’t killed each time a request
finishes but reused on next.

Hi All

New to the forum and new to nginx. Found out about it via the vBulletin
forum and am currently setting up a small server to run my vB powered
site with 100% nginx & php-fpm. I would love to know more about this
caching mechanism, never heard of it before. Are there any concrete
examples of its use, sounds perfect for me running on a 768Mb VPS.

The current setup I have runs nginx with 2 workers and 2048 connections
per worker. I have 20 php-fpm children running with Xcache which serves
the site OK until the connections increase. I hit the site with a
Loadimpact run and around 120 connection it started spewing out 404’s.
The only way to stop it was to restart php-fpm. So the facility above
sounds perfect for my limited memory setting…

Any help suggestions or pointers would be greatly appreciated. Just want
to dump apache as quick as I can!

Cheer
Bruce

Posted at Nginx Forum:

rahul286 · June 27, 2010, 3:46am

rahul286 Wrote:

cache levels after that…
-Rahu
Yes! wondering if they all work together. hoping that it will not
sublime it’s ops.

Posted at Nginx Forum:

rahul286 · June 27, 2010, 1:30am

Thanks All.
Moved to PHP-FPM and also removed (purged) unwanted php-extensions.

I am using php 5.3.2 and I read somewhere that pm = dynamic feature
should be used in PHP-FPM only if PHP > 5.3.3.

Should I wait , or go ahead with Reinis R. settings…

pm = dynamic
pm.max_children = 70
pm.start_servers = 20
pm.min_spare_servers = 5
pm.max_spare_servers = 20
pm.max_requests = 1000

I am also thinking of using nginx fastcgi_cache

But started to feel I will be having too many cache levels after
that…

nginx fastcgi_cache. apc, memcache and wordpress caching plugins.

Will they work all together smoothly or will become counter-productive?

Any suggestions?

Thanks again,
-Rahul

Posted at Nginx Forum:

rahul286 · September 9, 2011, 2:34pm

Hi,
what would you guys recommend to set for a website loaded heavily with
PHP and some Ajax features and around 2000 users online at the same time
?
Currently I have settings like:

PHP_FCGI_CHILDREN=10
PHP_FCGI_MAX_REQUESTS=1000

worker_connections  20392;
 multi_accept off;
 use epoll;
client_body_timeout 200;
client_header_timeout 100;.
client_max_body_size 40M;
client_body_buffer_size 1280k;
send_timeout 100;
limit_zone limit_per_ip $binary_remote_addr 180m;
limit_conn limit_per_ip 150;
limit_rate 2200k;
keepalive_timeout 15;

Using nginx/0.7.67 and php 5.2.17.

I cannot migrate to php 5.3.3, so I cannot use native php-fpm.
Should I try to tweak fast-cgi better or try to get PHP-FPM working ?
Would nginx upgrade help with performance ?
Website works but with many users it gets veryyyyyyyyy slow with
responses.
Thanks guys !

rahul286 · September 9, 2011, 3:38pm

I cannot migrate to php 5.3.3, so I cannot use native php-fpm.
Should I try to tweak fast-cgi better or try to get PHP-FPM working ?

You can apply FPM patches (it was a quite a long time before it got into
core) also to 5.2.x tree and for 5.2.17 (
http://php-fpm.org/downloads/php-5.2.17-fpm-0.5.14.diff.gz ).
The dynamic manager will let you adjust best process numbers and will
show the slow scripts.

rr

rahul286 · June 27, 2010, 3:59pm

Hi Eric,

Yes! wondering if they all work together. hoping that it will not
sublime it’s ops.

I decided NOT to use memcache. Memcache is good for distributed
environment.

I am using APC for php opcode cache and user-data as well.
In case of w3 total cache plugin, APC can also be used as page-cache
instead of hard-disk.

Now only confusion, I am in is nginx fastcgi_cache.
In theory, and from what I have read, if nginx fastcgi_cache serves
request, no load should be forwarded to php-fpm or php-fastcgi (in
short, php part).
I guess with APC also, by using it as page-cache, we are achieving
same thing. Not sure though!

-Rahul