Php-fastcgi and memory leaks

Further to the email below, I’m still having problems with PHP5 fastcgi
and nginx. I’m finding that php slowly takes up 100% of the available
memory and starts to drop connections at around 70%. At the moment I’ve
got monit restarting the fastcgi processes when the memory usage hits
70%, but this isn’t optimal as the scripts I’m running shouldn’t take
anywhere near that on a virtual server with 300MB dedicated ram.

I’m wondering whether anyone else has come across the same problem
running php5 fastcgi who has managed to find a fix or work-around?

Thats a common issue (at least for me). Although php leaks itself with
every
version less there are still quite a bunch of extensions outside that
dont
handle the memory bits the right way.

As a workarround I use this patch
http://devzone.zend.com/content/patch/pat38.txt

Its pretty old and made for php4 but with little tweaks you can apply
also
to php5 (I could make a diff and throw in here). The basic idea is you
have
an extra setting PHP_FCGI_MAX_RAM_MB - you can limit at what memory
usage
the php child should handle the last request and die (so the master
process
can spawn a new one - similar to PHP_FCGI_MAX_REQUESTS ).

This solution is better because you dont accidently kill a process which
is
in a middle of generating a response.

rr

----- Original Message -----
From: “Phillip B Oldham” [email protected]
To: [email protected]
Sent: Tuesday, March 25, 2008 11:44 AM
Subject: php-fastcgi and memory leaks

Thats interesting. A diff would be useful if you have one.

What’s the best way to set these settings? At the moment I’m exporting
them just before I use spawn-fcgi (from lighttpd), but even though I’ve
set it to a low number (10) I don’t think its being honoured. I’m
running CentOS5, if that’s any help.

If I can’t find a suitable work-around for this I may have to switch to
apache with php compiled in - something I don’t really want to do as I’m
finding working with nginx to be a great experience.

Reinis R. wrote:

what memory usage the php child should handle the last request and die
To: [email protected]

I’m wondering whether anyone else has come across the same problem
running php5 fastcgi who has managed to find a fix or work-around?

Phillip B Oldham
The Activity People
[email protected] mailto:[email protected]


Policies

This e-mail and its attachments are intended for the above named
recipient(s) only and may be confidential. If they have come to you in
error, please reply to this e-mail and highlight the error. No action
should be taken regarding content, nor must you copy or show them to
anyone.

This e-mail has been created in the knowledge that Internet e-mail is
not a 100% secure communications medium, and we have taken steps to
ensure that this e-mail and attachments are free from any virus. We must
advise that in keeping with good computing practice the recipient should
ensure they are completely virus free, and that you understand and
observe the lack of security when e-mailing us.

I’m using spawn-fcgi, from the lighttpd project. It seemed the easiest
way to manage the php instances. Is there a better way to spawn php
instances for nginx?

Yordan G. wrote:

WEB: http://gigavolt-bg.net/
Blog: http://live.gigavolt-bg.net/*

Phillip B Oldham
The Activity People
[email protected] mailto:[email protected]


Policies

This e-mail and its attachments are intended for the above named
recipient(s) only and may be confidential. If they have come to you in
error, please reply to this e-mail and highlight the error. No action
should be taken regarding content, nor must you copy or show them to
anyone.

This e-mail has been created in the knowledge that Internet e-mail is
not a 100% secure communications medium, and we have taken steps to
ensure that this e-mail and attachments are free from any virus. We must
advise that in keeping with good computing practice the recipient should
ensure they are completely virus free, and that you understand and
observe the lack of security when e-mailing us.

*Hello Phillip,

How to create you are fastcgi gataway?


Please excuse my bad english…

С уважение,
Й. Георгиев.

WEB: http://gigavolt-bg.net/
Blog: http://live.gigavolt-bg.net/*

Hi Phillip,

Do you mind sharing your monitrc section concerning the monitoring of
the spawn-fcgi process? I am currently unable to do so. This way I
could hammer my php apps with httperf and see if I also have this
memory leaks problem.

On Wed, Mar 26, 2008 at 9:12 AM, Phillip B Oldham

Thank you very much Phillip.

php-fastcgi (webserver)

--------------------

check process php-fastcgi with pidfile /var/run/php-fastcgi.pid
start program = “/etc/init.d/php-fastcgi start”
stop program = “/etc/init.d/php-fastcgi stop”
if cpu usage > 95% for 3 cycles then alert
if totalmem usage > 90% for 1 cycles then restart
depends php-fastcgi_init
depends php-fastcgi_bin

check file php-fastcgi_init with path /etc/init.d/php-fastcgi
group www

check file php-fastcgi_bin with path /usr/local/bin/php-fcgi
group www

And my php-fcgi script looks like this:

#!/bin/bash
PHP_FCGI_MAX_REQUESTS=25
export PHP_FCGI_MAX_REQUESTS
/usr/local/bin/spawn-fcgi -a 127.0.0.1 -p 10005 -u apache -g apache -C 3
-f /usr/local/bin/php-cgi -P /var/run/php-fastcgi.pid

Doesn’t look as though PHP_FCGI_MAX_REQUESTS is being honoured though,
which is annoying. I’ve even tried setting it to 1 (so the child dies
after every request) but it has no effect. I think in the whole time
I’ve been working with php under nginx in this way I’ve only seen a
php process once.

Thomas wrote:

recipient(s) only and may be confidential. If they have come to you in

Phillip B Oldham
The Activity People
[email protected] mailto:[email protected]


Policies

This e-mail and its attachments are intended for the above named
recipient(s) only and may be confidential. If they have come to you in
error, please reply to this e-mail and highlight the error. No action
should be taken regarding content, nor must you copy or show them to
anyone.

This e-mail has been created in the knowledge that Internet e-mail is
not a 100% secure communications medium, and we have taken steps to
ensure that this e-mail and attachments are free from any virus. We must
advise that in keeping with good computing practice the recipient should
ensure they are completely virus free, and that you understand and
observe the lack of security when e-mailing us.

I use and:

PHP_FCGI_CHILDREN=1000

export PHP_FCGI_CHILDREN

On Wed, Mar 26, 2008 at 4:18 PM, Corey D. [email protected] wrote:

On Wed, Mar 26, 2008 at 4:08 AM, Thomas [email protected] wrote:

Thank you very much Phillip.


Corey D.
http://www.atmos.org/
http://www.engineyard.com/


Please excuse my bad english…

С уважение,
Й. Георгиев.

WEB: http://gigavolt-bg.net/
Blog: http://live.gigavolt-bg.net/

Hey Guys,

I don’t know how much it’ll help, but we do things a little
differently for php-fcgi and monit. We’re not hosting custom apps as
often as open source things like wordpress, but I figured I’d share
our approach where I work. We tend to have decent results with hard
memory limits, not percentages.

http://pastie.caboo.se/170872

Out of interest, if I were to add the following to my nginx.conf:

env PHP_FCGI_MAX_REQUESTS=10;

Would that be picked up by php?

Rob S. wrote:

check process php-fastcgi with pidfile /var/run/php-fastcgi.pid
check file php-fastcgi_bin with path /usr/local/bin/php-fcgi
Doesn’t look as though PHP_FCGI_MAX_REQUESTS is being honoured

could hammer my php apps with httperf and see if I also have this

for nginx?

Phillip B Oldham
be taken regarding content, nor must you copy or show them to anyone.
security when e-mailing us. ________________________________

ensure that this e-mail and attachments are free from any virus. We
must advise that in keeping with good computing practice the
recipient should ensure they are completely virus free, and that you
understand and observe the lack of security when e-mailing us.


Phillip B Oldham
The Activity People
[email protected] mailto:[email protected]


Policies

This e-mail and its attachments are intended for the above named
recipient(s) only and may be confidential. If they have come to you in
error, please reply to this e-mail and highlight the error. No action
should be taken regarding content, nor must you copy or show them to
anyone.

This e-mail has been created in the knowledge that Internet e-mail is
not a 100% secure communications medium, and we have taken steps to
ensure that this e-mail and attachments are free from any virus. We must
advise that in keeping with good computing practice the recipient should
ensure they are completely virus free, and that you understand and
observe the lack of security when e-mailing us.

I am not sure if this is “Gentoo” specific but they use ALLOWED_ENV
settings when using spawn-fcgi here is the two relevant files

http://www.pastie.org/private/scbfzxfcvfqqdkohjdmw

Hi Phillip,

As promised I have been trying to smash my spawn-fcgi process with ab
but with no success. The cpu went crazy as expected, but the memory
remained low. I don’t if I am not testing enough. I was testing
retrieving the index page of a PunBB forum hosted on my server. With
too high concurrency, ab would stop benchmarking because of php-cgi
starting to fail connections. spawn-fcgi uses 5MB of memory and MySQL
21MB.

I am using Ubuntu 7.10 Server Edition with all official Ubuntu packages.

And thanks for sharing your monitrc file :wink:

No matter how I look at it this doesnt seem right:

First - why and how the heck are you able to “smash” ab against
spawn-fcgi
(or I do misunderstood something)? Because spawn-fcgi isnt a permanent
process nor does it handle any HTTP requests (or acts as fcgi gateway to
handle the requests from nginx) … spawn-fcgi is only an executable
provided
with lighttpd to spawn PHP master process (with setuid/setgid feature
when
the adaptive spawning on lighty was basically disabled).
You can fully do the same just with php executable (built as fastcgi) eg
provide port/socket and other settings from command line (and some from
ENV
variables).

So I can’t really get how can spawn-fcgi eat up ram because it shouldnt
be
on the process list at all…

Second - MySQL ram usage on the other hand is unbelievable small… but
thats
another discussion :slight_smile:

rr

----- Original Message -----
From: “Thomas” [email protected]
To: [email protected]
Sent: Saturday, March 29, 2008 11:42 PM
Subject: Re: php-fastcgi and memory leaks

You are right, I didn’t express myself correctly. One should read
“php-spawn-fcgi uses 5MB of memory”, php-spawn-fcgi being a daemon
that I created using the following command:

/usr/bin/spawn-fcgi -a 127.0.0.1 -p 10005 -u www-data -g www-data -C 5
-f /usr/bin/php-cgi -P /var/run/spawn-fcgi/php-spawn-fcgi.pid

I guess that Phillip is using the same kind of setup to handle php
with Nginx. And that the ram usage he is talking about is the ram
usage of the php-spawn-fcgi process.

The MySQL ram usage remains low certainly because I am always hitting
the same index page which has very little data on it (I created 10
empty sub-forums), but that’s benchmarking, it is impossible to
perfeclty mimic real-life behavior and anyway the goal was no to
stress test mysql.

Best regards,

Hi,

On Mon, Mar 31, 2008 at 12:55:16PM +0100, Phillip B Oldham wrote:

I’m confused - when using a command similar to the one below, is it
spawn-fcgi handling memory or php-cgi? When I check the processlist (ps
aux) I see 4 instances of “php-cgi”, each one with a slowly growing
memory usage.

If you use ‘ps auxf’, you’ll probably see that one of the php-cgi
processes is the master, with all others hanging off from it as its
children. By that time, spawn-fcgi is long gone (it called exec…() and
became the master php process). BTW, do you really see increasing memory
usage on all the processes including the parent?

spawn-fcgi isn’t really a process manager, it just starts your fastcgi
server and hopes for the best. Anyway, only the php interpreter is to
blame here.

Maybe its the app I’m using which doesn’t finish correctly. Its
contributed to by a few different developers, some of which are new to
the system and might not be finishing their scripts with the correct
return state.

It shouldn’t matter at all, the memory used for e.g. php code and
variables should get properly freed at the end of the request. You might
leak resources (database connections etc.) though. Are you using any
non-standard php extensions?

Best regards,
Grzegorz N.

Grzegorz N. wrote:

If you use ‘ps auxf’, you’ll probably see that one of the php-cgi
processes is the master, with all others hanging off from it as its
children. By that time, spawn-fcgi is long gone (it called exec…() and
became the master php process). BTW, do you really see increasing memory
usage on all the processes including the parent?

Yes, all processes, including the parent. I’ve set-up 3 children and one
parent, and all 4 instances slowly grow in size.

spawn-fcgi isn’t really a process manager, it just starts your fastcgi
server and hopes for the best. Anyway, only the php interpreter is to
blame here.

Thats what I thought.

It shouldn’t matter at all, the memory used for e.g. php code and
variables should get properly freed at the end of the request. You might
leak resources (database connections etc.) though. Are you using any
non-standard php extensions?

I don’t know about non-standard. I’m not using any PECL entensions, but
I have custom compiled the php instance on the box to include items such
as IMAP support and turn off things that we don’t use. My compile
options are below:

./configure
–disable-all
–disable-ipv6
–disable-short-tags
–disable-cli
–with-zend-vm=GOTO
–without-pear
–enable-inline-optimization
–enable-fast-install
–enable-fastcgi
–enable-discard-path
–enable-force-cgi-redirect
–enable-zend-multibyte
–enable-sigchild
–enable-libgcc
–enable-static
–enable-libxml
–enable-dom
–enable-mbstring
–enable-spl
–enable-gd-native-ttf
–enable-exif
–enable-pdo
–enable-session
–enable-simplexml
–enable-soap
–enable-sockets
–enable-zip
–enable-bcmath
–enable-ctype
–enable-json
–enable-posix
–enable-reflection
–enable-tokenizer
–enable-xml
–with-config-file-scan-dir=/etc/php5
–with-pcre-regex
–with-pcre-dir
–with-libxml-dir
–with-openssl
–with-kerberos
–with-zlib
–with-bz2
–with-curl
–with-curlwrappers
–with-gd
–with-jpeg-dir
–with-png-dir
–with-ttf
–with-freetype-dir
–with-gettext
–with-imap
–with-imap-ssl
–with-xmlrpc
–with-xsl
–with-iconv
–with-mysql=/usr/local/bin
–with-mysql-sock=/var/lib/mysql/mysql.sock
–with-pdo-mysql=/usr/bin/mysql_config
–with-gdbm
–with-imap
–with-imap-ssl

Don’t know if any of those items are known to cause memory leaks.

Phillip B Oldham
The Activity People
[email protected] mailto:[email protected]


Policies

This e-mail and its attachments are intended for the above named
recipient(s) only and may be confidential. If they have come to you in
error, please reply to this e-mail and highlight the error. No action
should be taken regarding content, nor must you copy or show them to
anyone.

This e-mail has been created in the knowledge that Internet e-mail is
not a 100% secure communications medium, and we have taken steps to
ensure that this e-mail and attachments are free from any virus. We must
advise that in keeping with good computing practice the recipient should
ensure they are completely virus free, and that you understand and
observe the lack of security when e-mailing us.

On Mon, Mar 31, 2008 at 01:28:22PM +0100, Phillip B Oldham wrote:

Yes, all processes, including the parent. I’ve set-up 3 children and one
parent, and all 4 instances slowly grow in size.

Now that is strange. The parent process doesn’t do almost anything.
Maybe running under valgrind will provide some insight (though it is
SLOW).

I don’t know about non-standard. I’m not using any PECL entensions, but
I have custom compiled the php instance on the box to include items such
as IMAP support and turn off things that we don’t use. My compile
options are below:

./configure \

(snip)

Don’t know if any of those items are known to cause memory leaks.

In theory, no. In practice, who knows. It is PHP, after all :wink:

AFAIK, there is no possibility (apart from very nasty hacks) for
extensions to cause gradual increase in parent’s memory usage.
Does the parent process grow at the same rate?

Is php-fcgi the only process that exhibits this behaviour? Maybe there’s
something wrong with your system libraries?

Frankly, I have no idea. If you can reproduce the problem in a test
environment, try running php under “valgrind --leak-check=full”

Best regards,
Grzegorz N.

I’m confused - when using a command similar to the one below, is it
spawn-fcgi handling memory or php-cgi? When I check the processlist (ps
aux) I see 4 instances of “php-cgi”, each one with a slowly growing
memory usage.

Maybe its the app I’m using which doesn’t finish correctly. Its
contributed to by a few different developers, some of which are new to
the system and might not be finishing their scripts with the correct
return state.

Thomas wrote:

Phillip B Oldham
The Activity People
[email protected] mailto:[email protected]


Policies

This e-mail and its attachments are intended for the above named
recipient(s) only and may be confidential. If they have come to you in
error, please reply to this e-mail and highlight the error. No action
should be taken regarding content, nor must you copy or show them to
anyone.

This e-mail has been created in the knowledge that Internet e-mail is
not a 100% secure communications medium, and we have taken steps to
ensure that this e-mail and attachments are free from any virus. We must
advise that in keeping with good computing practice the recipient should
ensure they are completely virus free, and that you understand and
observe the lack of security when e-mailing us.

On Mon, Mar 31, 2008 at 01:47:07PM +0100, Phillip B Oldham wrote:

Actually, thats my error - they were jumping around a lot in top. ps auxf shows the parent process holding steady at 1.7%.

Whew, I can retain my sanity :slight_smile:

Thanks, I’ll look into doing that. I’ve not used valgrind before, so
should be fun!

Yeah, lots of fun. Though if you’re leaking database connections or
something, you might see it e.g. on lsof output. Try disabling
persistent connections (at least temporarily) for the database you’re
using.

Best regards,
Grzegorz N.

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs