Weird freeze problem

Hello,

I am running mongrel 1.1 in production mode behind a lighttpd with
mod_proxy.
At first everything works fine, however, after a few days, the mongrel
servers stop answering at all, even if I try to connect to them
directly.
I have checked logs but there is no sign of anything at all, and even
when there was not a single request for a few days, it did freeze, and
that on repeated occasions. What should I do to be able to find out what
is the problem, or to solve it?

Thanks,


Best regards,
Paul-Kenji Cahier mailto:[email protected]

Hey Paul,

This sounds like the problem I’ve been hitting. The folks on this
group suggested updating the ActiveRecord timeout.

Apparently if one of your mongrels doesn’t get any play for some
amount of time, it automatically drops the database connection. This
in turn will cause your mongrel to become unresponsive.

Here’s the change I made in environment.rb:

Set the verification timeout so that the mongrel connection won’t

die if left alone (1 week)
ActiveRecord::Base.verification_timeout = 604800

So far so good, but I’ll know more as Iet it run for the next few
days. Hopefully there aren’t any issues with setting a timeout this
large. Shout back to the group if this fixes it for you.

Thanks,
Pete

PS. Prior to finding this fix, I’ve been using monit to restart the
servers when they become unresponsive.

Shouldn’t the ActiveRecord database adapter recover from a lost
connection
and reconnect? I had a quick look at the source and the adapter has
error
messages for lost connections and code to reconnect, but it doesn’t seem
to
be used anywhere in the adapter. (I only had a quick look in
mysql_adapter.rb so it may be used somewhere else.)

I’m not familiar enough with the code to try and fix it and don’t have
the
time right now. Perhaps someone who is running into the issue can spare
some
time to fix it and pass the fix back to the ActiveRecord developers?

On Nov 8, 2007 9:45 AM, Geoffrey C. [email protected]
wrote:

Shouldn’t the ActiveRecord database adapter recover from a lost connection
and reconnect? I had a quick look at the source and the adapter has error
messages for lost connections and code to reconnect, but it doesn’t seem to
be used anywhere in the adapter. (I only had a quick look in
mysql_adapter.rb so it may be used somewhere else.)

I’m not familiar enough with the code to try and fix it and don’t have the
time right now. Perhaps someone who is running into the issue can spare some
time to fix it and pass the fix back to the ActiveRecord developers?

This issue arise from time to time on every list where ActiveRecord is
a topic :stuck_out_tongue:
(Named mongrel, merb, ruby-talk).

Ezra’s suggestions posted a few days back on merb ml is handy:

Thread.new { loop { ActiveRecord::Base.verify_active_connections!;
sleep 60*60} }.priority = -10

I’m not fond by the big sleep, since I experienced bad results using
them. An alternative will be 1800.times { sleep 2 }

YMMV,


Luis L.
Multimedia systems

Leaders are made, they are not born. They are made by hard effort,
which is the price which all of us must pay to achieve any goal that
is worthwhile.
Vince Lombardi