First - all my servers are running fine. Thanks a ton to nginx! Lately,
I have been brainstorming a lot on 3 configuration variables to squeeze
maximum performance that I can get from my hardware.
3 variables of my interest are:
[list=1]
[] worker_processes
[] worker_connections
[*] PHP_FCGI_CHILDREN
[/list]
I searched extensively on Google, mailing lists, wiki, etc but till date
couldn’t find any robust guide as on how these affects performance and
other aspects of my webserver/webapp.
As I am aiming for very-very high traffic configuration, I will first
move all images/js/css load to CDN.
So my nginx will be mostly busy in dealing PHP stuff.
[list]
[] So will it make more sense to increase PHP_FCGI_CHILDREN or
worker_processes?
[] Is it good to make PHP_FCGI_CHILDREN = 1 if I am using APC
for PHP opcode caching?
[/list]
If I want to achieve number of 100,000, then what is difference between
following arithmetic:
[list]
[] 10 worker_processes * 10,000 worker_connections * 1
PHP_FCGI_CHILDREN
[] 10 worker_processes * 1,000 worker_connections * 10
PHP_FCGI_CHILDREN
[] 10 worker_processes * 100 worker_connections * 100
PHP_FCGI_CHILDREN
[] 1 worker_processes * 1000 worker_connections * 100
PHP_FCGI_CHILDREN
[*] 1 worker_processes * 100,000 worker_connections * 1
PHP_FCGI_CHILDREN
[/list]
I know this arithmetic is not straight-forward as 3 variable controls 3
different aspects. But how does they impact each other? How amount of
RAM, traffic pattern, PHP/non-PHP traffic ratio affects them?
In short, what are all parameters I must consider?
Sorry if some of my questions sounds lame. But I really wants to dig
deeper into nginx config for high traffic load.
Also if you think I am totally going in wrong direction, please feel
free to correct me.
Thanks,
-Rahul
Posted at Nginx Forum:
[*] Is it good to make PHP_FCGI_CHILDREN = 1 if I am using APC
for PHP opcode caching?
No.
One child won’t do anything good on a multicore box (giving the usuall
nature of php scripts - waiting on external resources like DBs etc). Nor
it
can saturate single cpu (also APC doesnt limit you here in any way).
I would suggest to give it a try to the new FPM sapi/manager which is
now
included also in 5.3.x core.
It gives you the ability to dynamically manage a pool of php processes
(spawn more php childs if the server gets loaded or kill the unneeded
spares). That way you dont need to control any settings from the
webservers
side.
http://php-fpm.org/
http://php-fpm.org/download/
rr
Thanks Reinis.
Regarding…
[*] Is it good to make PHP_FCGI_CHILDREN = 1 if I am using APC
for PHP opcode caching?
No.
I read somewhere that 2 PHP processes do not share APC cache. So if I
have 10 PHP_FCGI_CHILDREN - that means 10 times cache size.
On
I would suggest to give it a try to the new FPM sapi/manager which is now included also in 5.3.x core.
I do not use FPM and I really wonder why it is used?
We can directly use php-fastcgi as mentioned here
Thanks for your time buddy.
If you have more inputs, please share.
-Rahul
php-fpm is an actual management daemon, as opposed to using spawn-fcgi
which you are still responsible for controlling. If you’re doing the
spawn-fcgi route you might as well just use php-cgi -b and use upstart
or some other controlling daemon and just do it yourself. spawn-fcgi
doesn’t give you anything special AFAIK.
Rasmus himself uses PHP-FPM now and gave it basically a thumbs up (in
person) and nginx for wepay.com (the company he works for now)
You don’t get much better clout than that.
php-fpm is an actual management daemon…
Thanks for details regarding PHP-FPM
When I bumped into it first time, I thought its overhead - more like
some
people using lighthppd for running PHP on nginx.
I will definitely try it but I still looking for equation to set numbers
like worker_processes, worker_connections & PHP_FCGI_CHILDREN (or
PHP_FPM
counterpart) perfectly.
I read somewhere that 2 PHP processes do not share APC cache. So if I
have 10 PHP_FCGI_CHILDREN - that means 10 times cache size.
That could be true if the php processes are spawned seperatly (by
webserver)
each time a client request comes in - which is slow as hell (and
somewhat
mimics the apache forking mechanism).
The preferred way (especially for high concurrency) is to spawn the php
as
permanent process and the webserver (nginx) talks to it through fastcgi
(
Module ngx_http_fastcgi_module ) - (you just provide ip:port
which gives you also the ability if needed to use multiple boxes) that
way
all the php processes have the same APC opcode cache / shared memory
(and
there is no need to recompile script each time a new child is spawned).
Not
to mention other benefits like persistant connections (to memcache / db
/
etc) which aren’t killed each time a request finishes but reused on
next.
spawn-fcgi was kinda early way to do this (came from lighttpd) but now
the
inbuilt FPM sapi has much more control and options (like to kill
processes
which run for too long / monitor/backtrace the code) to miss the
opportunity
to try it out.
rr
Thanks Reinis for your time and valuable info.
The preferred way (especially for high concurrency) is to spawn the php as permanent process and
the webserver (nginx) talks to it through fastcgi ( Module ngx_http_fastcgi_module )
I have setup my nginx as mentioned here -
http://wiki.nginx.org/NginxHttpFcgiModule
And I guess for this reason my APC cache is working fine! 
spawn-fcgi was kinda early way to do this (came from lighttpd) but now the inbuilt FPM sapi has much more control and options …
I am really convinced about PHP-FPM and will definitely give it a try.
Specially “upstream” stuff.
The basic approach usually as far as I know is to set worker_processes based on your CPU core count …
In VPS, even with linode/slichost company, you see 4 cores when you
cat /proc/cpuinfo
But as its VPS and not PS, cores may be shared by some unknown fellows.
Do u still think, keeping core_count math still helps?
So in short - both of the software products are clever enough to tell you what could be the optimal setting for you based on the workload of the box.
Agree. Switching to nginx was one of my best decision.
Fellow bloggers envy my servers uptime and speed and the fact is I pay
less in terms of hosting bill 
remember also to tweak other system settings - like maximum open file descriptors, tcp port range etc
I will check this out. Looks like I never traveled in this direction.
Thanks again Reinis.
-Rahul
On Tue, Jun 15, 2010 at 12:37 PM, Rahul B. [email protected]
wrote:
I have setup my nginx as mentioned here -
Module ngx_http_fastcgi_module
And I guess for this reason my APC cache is working fine!
I believe (and from what I see and hear) the APC cache is shared by
all children from the parent.
So each pool can share. I don’t think APC is global cross all pools
however it might be, because when I do apc_info() or whatever the
function is, I see other people’s files (in different pools) in my
file cache.
@Michael
Thanks again for heads up on PHP-FPM
I am reading your post -
http://michaelshadle.com/2010/05/21/php-fpm-and-nginx-upstart-scripts/
I hope to see our server using PHP-FPM next month (with ubuntu 10
+wordpress
3)
-Rahul
I will definitely try it but I still looking for equation to set numbers
like worker_processes, worker_connections & PHP_FCGI_CHILDREN (or PHP_FPM
counterpart) perfectly.
The basic approach usually as far as I know is to set worker_processes
based
on your CPU core count (either identical or sometimes there is benefit
setting with some multiplier - 2*cores (I have seen some improvements
when
using nginx as on-the-fly image resizer to utilise the box more)).
worker_connection - typical 1024 or 2048 usually is enough. Bassically
the
math is based on max_clients where: max_clients = worker_processes *
worker_connections
Anyways nginx should complain in error log if the settings are too low
and
all the workers are exausted (like you have thousands of
keepalive/active
connections).
The same with FPM - the basic settings which come with the sample config
should be enough. If you leave the default notice error level log FPM
should
tell you if all of the php pool is fully used and there is a need to
increase the setting.
So in short - both of the software products are clever enough to tell
you
what could be the optimal setting for you based on the workload of the
box.
p.s. remember also to tweak other system settings - like maximum open
file
descriptors, tcp port range etc
rr
So each pool can share. I don’t think APC is global cross all pools
however it might be, because when I do apc_info() or whatever the
function is, I see other people’s files (in different pools) in my
file cache.
In my case my VPS is hosting few domains with single user www-data.
PHP, webroot, files all are owned as well as forked by this single user.
I guess u mean apc.shm_segments by pools?
I have declared 4 pools but apc.php is saying…
Shared Memory = 1 Segment(s) with 60.0 MBytes (mmap memory, pthread
mutex
locking)
U can see my apc stats here - http://devilsworkshop.org/apc.php
I hope there is harm in making cache stats public!
-Rahul
2010/6/15 Reinis R. [email protected]:
math is based on max_clients where: max_clients = worker_processes *
worker_connections
this is right only if your webserver is ONLY serving dynamic content
without caching. If nginx is serving static file or dynamic content
from cache, you will have less connections to the fcgi backend than
you have incoming connections on the web server. There is no way to
know exactly how many fcgi concurent connexions. You can set
max_children depending on your RAM. If you have 2Go of RAM, you’ll
want to keep a part of this memory for system caching (let say 500
Mo). You’ll keep some for the system and the others apps (nginx) (let
say 256Mo). It’ll stay a bit more than 1Go of RAM dedicated to fcgi.
Depending on what your scripts are doing, you can imagine a memory
size for one fcgi process and you can then deduce the max_children.
If you think your PHP process won’t consume more than 20Mo, you can
set max_children to 1256 (Mo) / 20 (Mo) =~ 62.
But it’s all depending on what your application is doing, how the
audition is surfing the website, you system, …
Hi jerome.
With your additional inputs, Can I safely set following rules for
personal
use?
Of course, any configuration needs monitoring/tweaking. But I think its
better to start with a base config which will be closer to the optimal
config.
*Now for…
worker_connections
*1024/2048 are safe values. So its better if I leave them untouched.
worker_processes
Number of CPU core or its multiplier (assuming u have dynamic
application
with decent caching)
More process ==>> Less Dynamic and Lower PHP_FCGI_CHILDREN
Less Process ==>> More Dynamic and Higher PHP_FCGI_CHILDREN
*PHP_FCGI_CHILDREN
*Using TOP command and looking at RES memory value.
Finding memory required by single instance of PHP. (in my case its 40MB
and
VIRT is 100MB!)
Then doing arithmetic like:
[Available RAM] = (Actual RAM) - (100 M or more OS + 200 M for DB + 16M
for
nginx (I have 8 worker_process with 2MB memory for each ))
Then,
PHP_FCGI_CHILDREN = [Available RAM] / Single PHP instances’ memory
requirement
Sorry for trying to get mechanical here.
But I already started to have better insight into all configuration
variables I was curious at.
Also are these numbers in anyway gets affected by fastcgi_cache -
http://wiki.nginx.org/NginxHttpFcgiModule#fastcgi_cache
Does fastcgi_cache reduces load on fascgi-php ?
worker_connections
this is right only if your webserver is ONLY serving dynamic content
without caching. If nginx is serving static file or dynamic content
from cache, you will have less connections to the fcgi backend than
you have incoming connections on the web server. There is no way to
know exactly how many fcgi concurent connexions. You can set
max_children depending on your RAM.
Nginx has a quite small memory footprint (if you dont go overboard with
buffers per connection).
And you can have thousands of open connections because of effective
event
models (especially when using it for comet-type applications) and still
not
consuming much memory.
For nginx-type server maximum clients doesnt mean the same as for apache
MaxClients which could result in the same number as httpd processes
running
(each with php module loaded) consuming loads of memory so you had to be
extra carefull with that setting otherways things could turn ugly.
The beauty about php in fastcgi mode is the controlled memory
consumption
(besides the typical leaking in php/extensions - which is “fixed” by
periodically restarting the childs (done by the master php process)) -
it
doesnt really change/jump - eg you set 10 startup childs which easily
can
handle more than just 10 users as nginx makes a queue and will pass the
requests to the php fcgi backend (of course if you are running php code
like
<? sleep(60); ?> then only 10 clients at time (minute) will be processed
for
dynamic content).
In short max_clients/worker_connections are only loosely related (it
doesnt
matter if thats a static or dynamic request - all go into the same
worker
threads (there is a bit difference if you make a lot of backend
connections/proxying)) to amount of php processes in a world of
nginx-type
servers.
And in case you use FPM even the variable PHP_FCGI_CHILDREN has become
deprecated as the new syntax for process management is something like:
pm = dynamic
pm.max_children = 70
pm.start_servers = 20
pm.min_spare_servers = 5
pm.max_spare_servers = 20
pm.max_requests = 1000
p.s. afaik the only thing which the php-dev team havent still
implemented is
the FCGI_OVERLOADED (according to fcgi spec) feature… to let the
webserver
know sooner that there aren’t enough resources on the backend to process
the
request rather than after timeouts.
rr
2010/6/15 Rahul B. [email protected]:
1024/2048 are safe values. So its better if I leave them untouched.
depends on your visitors load. You can go higher than that if needed.
But I don’t think it’s you case
worker_processes
Number of CPU core or its multiplier (assuming u have dynamic application
with decent caching)
More process ==>> Less Dynamic and Lower PHP_FCGI_CHILDREN
Less Process ==>> More Dynamic and Higher PHP_FCGI_CHILDREN
worker_process = num of CPU. nothing more
requirement
you can save more of OS as there is file caching which is usefull
Sorry for trying to get mechanical here.
it’s a good start, then you’ll tweak
But I already started to have better insight into all configuration
variables I was curious at.
Also are these numbers in anyway gets affected by fastcgi_cache -
Module ngx_http_fastcgi_module
Does fastcgi_cache reduces load on fascgi-php ?
Oh hell yes !
a dynamic content nginx is serving from cache won’t generate any load
on the fcgi side as it won’t go there
@ Reinis
And in case you use FPM even the variable PHP_FCGI_CHILDREN has become
deprecated as the new syntax for process management is something like…
and @ jerome
make pm.start_servers greater than pm.min_spare_servers and lower than
pm.max_spare_servers
I am not using FPM as of now.
But I will keep configuration given by you guys with me and will
definitely
use it while moving to FPM.
@Piotr
php is compiled only with what I need, nothing more.
That explains why your PHP is running with 50% less memory as compared
to
mine php process.
I will also try removing unwanted php extensions.
-Rahul
the FCGI_OVERLOADED (according to fcgi spec) feature… to let the webserver
know sooner that there aren’t enough resources on the backend to process the
request rather than after timeouts
My purpose of starting this thread is the same as FCGI_OVERLOADED.
I want to tweak my server before it gets overloaded.
Thanks to nginx, I haven’t faced any downtime in months. 
(Apart from one in a while timeout alerts I get form uptime monitoring
sites)
2010/6/15 Reinis R. [email protected]:
worker_connections
And you can have thousands of open connections because of effective event
periodically restarting the childs (done by the master php process)) - it
threads (there is a bit difference if you make a lot of backend
pm.max_spare_servers = 20
pm.max_requests = 1000
make pm.start_servers greater than pm.min_spare_servers and lower than
pm.max_spare_servers
because at startup, if no connection arrives and if start_servers is
lower than min_spare_servers fpm will kill one process per second
until current process count reaches min_spare_servers.
p.s. afaik the only thing which the php-dev team havent still implemented is
the FCGI_OVERLOADED (according to fcgi spec) feature… to let the webserver
know sooner that there aren’t enough resources on the backend to process the
request rather than after timeouts.
interesting. Does nginx handle this fcgi response ? You can open a bug
on bugs.php.net to request this feature and, if you have so, propose a
corresponding patch.
2010/6/15 Rahul B. [email protected]:
By the way I have memcached + APC + CDN + wp-super-cache and now exploring
fastcgi_cahce by nginx.
no i mean memory used by the kernel to cache local files in order not
to access file on hardrive. see output of free, the column “cached”
Hi jerome
All your points duly noted.
Revising all again…
*worker_connections = 2048
*I have its value 1024 (nginx default I guess)
I have one very important question here.
When we say server load - we mean number of HTTP requests. right?
Number of unique pageviews and other sweet number shown by Google
Analytic
wont be helpful, atleast directly.
Is setting up this number very high - like 20480 can adversely affect a
high
traffic server?
worker_process = num of CPU. nothing more
I will make this a law for myself.
you can save more of OS as there is file caching which is useful
By file caching do u mean php cache which stores page output in html
files
and then serve them via nginx rewrites.
Like wp-super-cache in wordpress world.
By the way I have memcached + APC + CDN + wp-super-cache and now
exploring
fastcgi_cahce by nginx.
I am just obsessed with performance.
And I am really not doing this to save my hosting bill.
I just want to sustain on single node as long as possible, though some
day,
our growth forces us to explore path of load-balancing, proxy servers!
a dynamic content nginx is serving from cache won’t generate any load
on the fcgi side as it won’t go there
This sounds nice.
Surprisingly I haven’t seen more about it?
-Rahul