Application stops working over night: (Errno::EPIPE (Broken


#1

I know this question gets asked a lot but I can’t seem to find the
answer when I search through the mailing list. First off here’s my set
up.

Cent OS 4.3
Apache 2.0
Fast CGI 2.4.0
Rails 1.1
Mysql 5.0

The app starts up fine, but every morning it refuses to start and gives
me the dreaded Application error untill I reload Apache. I thought my
database was timing out so I added this line to my environment.rb

ActiveRecord::Base.verification_timeout = 3000

My database is set to drop inactive connections after 2 hours, so my
database should keep the connection open. I can watch the processes on
mysql and the rails connection is dropped after about 10 minutes. If I
try to access the web page after this timeout, the app re-connects
perfectly. This is why I’m completely confused about why the app
refuses to start each morning. Here is the related lines from my
production log.

Errno::EPIPE (Broken pipe):
/usr/lib/ruby/1.8/mysql.rb:1042:in flush' /usr/lib/ruby/1.8/mysql.rb:1042:inwrite’
/usr/lib/ruby/1.8/mysql.rb:462:in write' /usr/lib/ruby/1.8/mysql.rb:436:incommand’
/usr/lib/ruby/1.8/mysql.rb:307:in stat' /vendor/rails/activerecord/lib/active_record/connection_adapters/mysql_adapter.rb:147:inactive?’
/vendor/rails/activerecord/lib/active_record/connection_adapters/abstract_adapter.rb:85:in
verify!' /vendor/rails/activerecord/lib/active_record/connection_adapters/abstract/connection_specification.rb:99:inverify_active_connections!’
/vendor/rails/activerecord/lib/active_record/connection_adapters/abstract/connection_specification.rb:98:in
verify_active_connections!' /vendor/rails/railties/lib/dispatcher.rb:70:inprepare_application’
/vendor/rails/railties/lib/dispatcher.rb:37:in dispatch' /vendor/rails/railties/lib/fcgi_handler.rb:150:inprocess_request’
/vendor/rails/railties/lib/fcgi_handler.rb:54:in process!' /usr/lib/ruby/gems/1.8/gems/fcgi-0.8.6.1/./fcgi.rb:600:ineach_cgi’
/usr/lib/ruby/gems/1.8/gems/fcgi-0.8.6.1/./fcgi.rb:597:in each_cgi' /vendor/rails/railties/lib/fcgi_handler.rb:53:inprocess!’
/vendor/rails/railties/lib/fcgi_handler.rb:23:in `process!’
/var/www/my_app/htdocs/mailing_lists/public/dispatch.fcgi:24

Charlie B.
Programmer
Castle Branch Inc.


#2

anyone? I’ve almost got my bosses convinced to use Rails!

On Thu, 2006-05-25 at 08:55 -0400, Charlie B. wrote:

The app starts up fine, but every morning it refuses to start and gives
refuses to start each morning. Here is the related lines from my
/vendor/rails/activerecord/lib/active_record/connection_adapters/abstract/connection_specification.rb:99:in `verify_active_connections!’

removed_email_address@domain.invalid
http://lists.rubyonrails.org/mailman/listinfo/rails

Charlie B.
Programmer
Castle Branch Inc.


#3

If it were my call, I would switch, but it’s not so I’m forced into
finding out how to keep the app from dieing every 4 hours. It must be
possible, I’ve been running a rails app on dreamhost for 6 monghts with
no problem.


#4

Charlie B. wrote:

If it were my call, I would switch, but it’s not so I’m forced into
finding out how to keep the app from dieing every 4 hours. It must be
possible, I’ve been running a rails app on dreamhost for 6 monghts with
no problem.

I don’t have the exact URL handy, but google this:

rails mysql timeout 14400

and you’ll see folks with the problem & answer. There may be
discussions in this forum with those words.


#5

Hi !

On May 25, 2006, at 5:55 AM, Charlie B. wrote:

The app starts up fine, but every morning it refuses to start and
gives
me the dreaded Application error untill I reload Apache. I thought my
database was timing out so I added this line to my environment.rb

ActiveRecord::Base.verification_timeout = 3000

> > Charlie B. > Programmer > Castle Branch Inc.

Charlie-

Apache and fcgi just don't seem to get along that well. I have never

been able to get rid of all the various errors you get when apache
and fcgi stop talking to each other. mod_fastcgi is just a little bit
crusty it seems. You may want to try using mod_fcgid as I have heard
that is a better option. But what I really recommend right now is to
dump fcgi. Upgrade your apache install to version 2.2 and use
mod_proxy_balancer to proxy to a few mongrel backends. Mongrel is
every bit as fast and way more stable then fcgi. There are a few good
articles around for the config options for this. But Bradley Taylor
has written a very nice config for using apache this way that lets
apache serve all statis and cached content , and then mongrel serves
all dynamic requests.

Here is what  vhost would look like if you were running 3 mongrel

backends.

<VirtualHost *:80>
ServerName yourdomain.com
DocumentRoot /var/apps/your_app/current/public

<Directory “/var/apps/your_app/current/public”>
Options FollowSymLinks
AllowOverride None
Order allow,deny
Allow from all

Configure mongrel_cluster

<Proxy balancer://mongrel_cluster>
BalancerMember http://127.0.0.1:8000
BalancerMember http://127.0.0.1:8001
BalancerMember http://127.0.0.1:8002

RewriteEngine On

Uncomment for rewrite debugging

#RewriteLog logs/your_app_deflate_log deflate
#RewriteLogLevel 9

Check for maintenance file and redirect all requests

RewriteCond %{DOCUMENT_ROOT}/system/maintenance.html -f
RewriteCond %{SCRIPT_FILENAME} !maintenance.html
RewriteRule ^.*$ /system/maintenance.html [L]

Rewrite index to check for static

RewriteRule ^/$ /index.html [QSA]

Rewrite to check for Rails cached page

RewriteRule ^([^.]+)$ $1.html [QSA]

Redirect all non-static requests to cluster

RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f
RewriteRule ^/(.*)$ balancer://mongrel_cluster%{REQUEST_URI}
[P,QSA,L]

Deflate

AddOutputFilterByType DEFLATE text/html text/plain text/xml
BrowserMatch ^Mozilla/4 gzip-only-text/html
BrowserMatch ^Mozilla/4.0[678] no-gzip
BrowserMatch \bMSIE !no-gzip !gzip-only-text/html

Uncomment for deflate debugging

#DeflateFilterNote Input input_info
#DeflateFilterNote Output output_info
#DeflateFilterNote Ratio ratio_info
#LogFormat ‘"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%
%)’ deflate
#CustomLog logs/your_app_deflate_log deflate

ErrorLog logs/your_app_error_log
CustomLog logs/your_access_log combined

Hope that helps. It really is a much more stable setup then apace and
fcgi. I highly recommend it.

Cheers-
-Ezra


#6

http://mongrel.rubyforge.org/faq.html:
_
_Q: Mongrel stops working if it’s left alone for a long time.

If you find that Mongrel stops working after a long idle time and you’re
using MySQL then you’re hitting a bug in the MySQL driver that doesn’t
properly timeout connections. What happens is the MySQL server side of
the connection times out and closes, but the MySQL client doesn’t
detect this and just sits there.
What you have to do is set:
|ActiveRecord::Base.verification_timeout = 14400|
Or to any value that is lower than the MySQL server’s
interactive_timeout setting. This will make sure that ActiveRecord
checks the connection often enough to reset the connection.