Php-fpm requests eventually goes to queue!

Hello,

 We're using nginx + php-fpm. Please check the following 

configurations
in php-fpm and sysctl in order to handle large amount of php-fpm
request,
but still 1000+ requests are getting into queue every 15min.

php-fpm.d/stats.conf

[stats]
listen = 127.0.0.1:9000
user = apache
group = apache
request_slowlog_timeout = 5s
slowlog = /var/log/php-fpm/stats-slow.log
listen.allowed_clients = 127.0.0.1
pm = dynamic
pm.max_children = 250
pm.start_servers = 40
pm.min_spare_servers = 20
pm.max_spare_servers = 40
pm.max_requests = 40000
listen.backlog = -1
request_terminate_timeout = 300s
rlimit_files = 13107200
rlimit_core = unlimited
env[HOSTNAME] = $HOSTNAME
env[TMP] = /tmp
env[TMPDIR] = /tmp
env[TEMP] = /tmp
pm.status_path = /status

The main config parameters of sysctl.conf :

vm.overcommit_memory = 1
fs.file-max = 7000000
net.ipv4.tcp_max_syn_backlog = 70000
net.core.netdev_max_backlog = 4096
net.core.somaxconn=65535
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 1
net.ipv4.ip_local_port_range = 1024 65000
net.ipv4.tcp_tw_reuse = 1

/etc/security/limits.conf
root soft nofile 700000
root hard nofile 700000

We’ve 72G of Ram and also writing on disk is 100+MB/s on Sas drives
which
makes high io util% most of the time.

Any clue why requests are still getting into php-fpm queue and max
children also reached errors occuring, even max_children are 250 *
40000. ??

Php-fpm status :

pool: stats
process manager: dynamic
start time: 22/May/2014:12:17:39 +0500
start since: 1140
accepted conn: 228244
listen queue: 579
max listen queue: 1970
listen queue len: 65535
idle processes: 167
active processes: 9
total processes: 176
max active processes: 250
max children reached: 1

Regards.
Shahzaib

This does not seem like an nginx issue?

These are the new status for php-fpm now :

pool: stats
process manager: dynamic
start time: 22/May/2014:12:17:39 +0500
start since: 3975
accepted conn: 866645
listen queue: 0
max listen queue: 2163
listen queue len: 65535
idle processes: 153
active processes: 2
total processes: 155
max active processes: 250
max children reached: 4

On Thu, May 22, 2014 at 12:37 PM, shahzaib shahzaib

Anyone ?

Following is the nginx config :

user nginx;

no need for more workers in the proxy mode

worker_processes 16;
error_log /var/log/nginx/error.log crit;
worker_rlimit_nofile 409600;
events {
worker_connections 10240; # increase for busier servers
use epoll; # you should use epoll here for Linux kernels 2.6.x
}
http {
server_name_in_redirect off;
server_names_hash_max_size 10240;
server_names_hash_bucket_size 1024;
include mime.types;
default_type application/octet-stream;
server_tokens off;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 5;
gzip on;
gzip_vary on;
gzip_disable “MSIE [1-6].”;
gzip_proxied any;
gzip_http_version 1.1;
gzip_min_length 1000;
gzip_comp_level 6;
gzip_buffers 16 8k;

You can remove image/png image/x-icon image/gif image/jpeg if you have

slow CPU
gzip_types text/plain text/xml text/css application/x-javascript
application/xml application/xml+rss text/javascript
application/atom+xml;
ignore_invalid_headers on;
client_header_timeout 3m;
client_body_timeout 3m;
send_timeout 3m;
reset_timedout_connection on;
connection_pool_size 256;
client_header_buffer_size 256k;
large_client_header_buffers 4 256k;
client_max_body_size 200M;
client_body_buffer_size 128k;
request_pool_size 32k;
output_buffers 4 32k;
postpone_output 1460;
client_body_in_file_only on;
log_format bytes_log “$msec $bytes_sent .”;
include “/etc/nginx/conf.d/virtual.conf”;
}

Hello!

On Thu, May 22, 2014 at 04:03:37PM +0500, shahzaib shahzaib wrote:

Anyone ?

[…]

On Thu, May 22, 2014 at 1:27 PM, Lord N. [email protected] wrote:

This does not seem like an nginx issue?


Maxim D.
http://nginx.org/

Met you after a long time maxim :slight_smile: how’s everything ?

You think, issue is not related to nginx but nginx and php-fpm work
together, that is the reason i posted the question on nginx forums.

pm.max_children = 250
pm.start_servers = 40
pm.max_requests = 40000

Any clue why requests are still getting into php-fpm queue and max
children also reached errors occuring, even max_children are 250 * 40000.
??

Your math is wrong - there is no such 250 * 40000, because
pm.max_requests
is just a number of requests after which the php child restarts (to
avoid
possible memory leak etc (it may also be 0)).

Your (maximum simultaneously running) concurrent requests are still 250
(~pm.max_children), so bassically if your request takes longer than 1
second
and there are more than 250 requests per second you will end up with a
queue.

You either speed up the php code (and everything besides it) / increase
the
FPM pool (more children or more pools/fpm backends) or use caching like
some
frontend cache (varnish/squid etc) or nginx with fastcgi_cache.

rr

Thanks, my math is not wrong my concept about php-fpm was wrong. I was
thinking each child can handle 40000 requests/sec. :frowning:

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs