Forum: Ruby on Rails mysql server has gone away?

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
38a8230ed3d5c685558b4f0aad3fc74b?d=identicon&s=25 joevandyk (Guest)
on 2005-11-29 11:26
(Received via mailing list)
What on earth would cause the following error?  I have multiple sites
using the same lighttpd and msyql and they are working just fine.

ActiveRecord::StatementInvalid (Mysql::Error: MySQL server has gone
away: SELECT * FROM contents WHERE (contents.`key` = 'index' )  LIMIT
1):
    /vendor/rails/activerecord/lib/active_record/connection_adapters/abstract_adapter.rb:78:in
`log'
    /vendor/rails/activerecord/lib/active_record/connection_adapters/mysql_adapter.rb:185:in
`execute'
    /vendor/rails/activerecord/lib/active_record/connection_adapters/mysql_adapter.rb:331:in
`select'
    /vendor/rails/activerecord/lib/active_record/connection_adapters/mysql_adapter.rb:176:in
`select_all'
    /vendor/rails/activerecord/lib/active_record/base.rb:445:in
`find_by_sql'
    /vendor/rails/activerecord/lib/active_record/base.rb:409:in `find'
    /vendor/rails/activerecord/lib/active_record/base.rb:407:in `find'
    /vendor/rails/activerecord/lib/active_record/deprecated_finders.rb:22:in
`find_first'
    /vendor/rails/activerecord/lib/active_record/base.rb:996:in `send'
    /vendor/rails/activerecord/lib/active_record/base.rb:996:in
`method_missing'
    /app/controllers/contents_controller.rb:12:in `show'
    /vendor/rails/actionpack/lib/action_controller/base.rb:854:in `send'
    /vendor/rails/actionpack/lib/action_controller/base.rb:854:in
`perform_action_without_filters'
    /vendor/rails/actionpack/lib/action_controller/filters.rb:332:in
`perform_action_without_benchmark'
    /vendor/rails/actionpack/lib/action_controller/benchmarking.rb:69:in
`perform_action_without_rescue'
    /vendor/rails/actionpack/lib/action_controller/benchmarking.rb:69:in
`measure'
    /vendor/rails/actionpack/lib/action_controller/benchmarking.rb:69:in
`perform_action_without_rescue'
    /vendor/rails/actionpack/lib/action_controller/rescue.rb:82:in
`perform_action'
    /vendor/rails/actionpack/lib/action_controller/base.rb:369:in `send'
    /vendor/rails/actionpack/lib/action_controller/base.rb:369:in
`process_without_session_management_support'
    /vendor/rails/actionpack/lib/action_controller/session_management.rb:116:in
`process'
    /vendor/rails/railties/lib/dispatcher.rb:38:in `dispatch'
    /vendor/rails/railties/lib/fcgi_handler.rb:141:in `process_request'
    /vendor/rails/railties/lib/fcgi_handler.rb:53:in `process!'
    /vendor/rails/railties/lib/fcgi_handler.rb:52:in `each_cgi'
    /usr/local/lib/ruby/gems/1.8/gems/fcgi-0.8.6.1/./fcgi.rb:597:in
`each'
    /usr/local/lib/ruby/gems/1.8/gems/fcgi-0.8.6.1/./fcgi.rb:597:in
`each_cgi'
    /vendor/rails/railties/lib/fcgi_handler.rb:52:in `process!'
    /vendor/rails/railties/lib/fcgi_handler.rb:22:in `process!'
    /home/joevd/cisv/current/public/dispatch.fcgi:24
Ef4d1b20eba4b45b4a477c33bae1d156?d=identicon&s=25 ksibilev (Guest)
on 2005-11-30 20:45
(Received via mailing list)
For example, when you fork a child ruby interpreter. On exit, it closes
mysql
socket.

Kent.
38a8230ed3d5c685558b4f0aad3fc74b?d=identicon&s=25 joevandyk (Guest)
on 2005-11-30 22:43
(Received via mailing list)
I ain't forking anything, as far as I know.
5ceea097a3b29cb6a5da6705926410f4?d=identicon&s=25 gerret.apelt (Guest)
on 2005-12-09 11:30
(Received via mailing list)
Hi Kent,

I saw your answer to Joe (below) and hope you can shed light on a
similar issue I'm having. I need to make AR calls in my main process
as well as in a forked process simultaneously. It looks something like
this:

def something
  fork do
    emails.each { |email|
      email.send  # calls SMTP
      email.log      # calls ActiveRecord subclass
    }
  end

  arInstance.do_something # do a bunch of AR stuff
end

With the above code I get an ActiveRecord::StatementInvalid error
(sample stacktrace below). Not always, but regularly, and I can't
narrow down the circumstances that make it happen.

You write that the DB socket is closed when the forked process exits.
So I'm guessing that is my problem.

But how to work around that?

I tried this:

fork do
  at_exit {
          ActiveRecord::Base.establish_connection
  }
end

which didnt help.

Any ideas from anybody are much appreciated -- I'm stumped. How can I
do AR work in a forked process without borking my application?

cheers
Gerret

Sample Stacktrace (edgerails):
-------------------------

ActiveRecord::StatementInvalid: Mysql::Error: : SELECT COUNT(*) FROM
logged_emails
    /home/gerret/railswork/sosamerica/vendor/rails/activerecord/lib/active_record/connection_adapters/abstract_adapter.rb:88:in
`log'
    /home/gerret/railswork/sosamerica/vendor/rails/activerecord/lib/active_record/connection_adapters/mysql_adapter.rb:181:in
`execute'
    /home/gerret/railswork/sosamerica/vendor/rails/activerecord/lib/active_record/connection_adapters/mysql_adapter.rb:327:in
`select'
    /home/gerret/railswork/sosamerica/vendor/rails/activerecord/lib/active_record/connection_adapters/mysql_adapter.rb:176:in
`select_one'
    /home/gerret/railswork/sosamerica/vendor/rails/activerecord/lib/active_record/connection_adapters/abstract/database_statements.rb:16:in
`select_value'
    /home/gerret/railswork/sosamerica/vendor/rails/activerecord/lib/active_record/base.rb:518:in
`count_by_sql'
    /home/gerret/railswork/sosamerica/vendor/rails/activerecord/lib/active_record/base.rb:511:in
`count'
    test/unit/email_batch_sender_test.rb:60:in `test_emails_get_logged'
Ce60c4f78a63b0695e4dafc4bd7964f7?d=identicon&s=25 vanek (Guest)
on 2005-12-09 12:59
(Received via mailing list)
somebody posted their solution to this problem a few days ago.
wouldn't hurt to give it a shot:

  def something
    begin
      fork do
        emails.each { |email|
          email.send  # calls SMTP
          email.log      # calls ActiveRecord subclass
        }
      end

      arInstance.do_something # do a bunch of AR stuff

    rescue ActiveRecord::StatementInvalid => e
      if e.to_s =~ /away/
             ActiveRecord::Base.establish_connection and retry
      else
             raise e
      end
    end
  end

OR, perhaps

  def monitor_connection
    begin
	yield

    rescue ActiveRecord::StatementInvalid => e
      if e.to_s =~ /away/
             ActiveRecord::Base.establish_connection and retry
      else
             raise e
      end
    end
  end


  monitor_connection do
      fork do
        emails.each { |email|
          email.send  # calls SMTP
          email.log      # calls ActiveRecord subclass
        }
      end

      arInstance.do_something # do a bunch of AR stuff
  end

OR maybe

   def something
      fork do
        emails.each { |email|
          email.send  # calls SMTP
          monitor_connection { email.log }
        }
      end

      monitor_connection do
         arInstance.do_something
      end
   end




Gerret Apelt wrote:

>       email.send  # calls SMTP
>
>   }
> end
>
> which didnt help.
>
> Any ideas from anybody are much appreciated -- I'm stumped. How can I
> do AR work in a forked process without borking my application?
>
> cheers
> Gerret
>
[snip]
5ceea097a3b29cb6a5da6705926410f4?d=identicon&s=25 gerret.apelt (Guest)
on 2005-12-09 13:53
(Received via mailing list)
Lou, thanks much. I hadn't caught the previous thread with Julik's
workaround or the discussion on bug #428, only read them just now.

Your monitor_connection {...} method does fix my unit test. Great. The
special nature of the whole forking business still poses a problem
though, because I cannot know in advance when the forked process will
terminate and take down the DB connection. I would have to wrap any
code that calls AR in a monitor_connection block, and I really don't
want to do that. Firing a dummy statement in a before_filter doesn't
sound too good either.

Reading through the bug discussionI'm not sure that my issue is
exactly the same as that addressed by bug #428. I'm running on
edgerails and the ticket's status is 'fixed'. Yet without the
monitor_connection workaround, I keep getting those StatementInvalid
exceptions.

Has the fix in edgerails taken care of this problem for everybody
else? Is it possible that the bugfix in edgerails takes care of timed
out connections, but not connections disrupted by an exiting subshell?

cheers
Gerret
Ce60c4f78a63b0695e4dafc4bd7964f7?d=identicon&s=25 vanek (Guest)
on 2005-12-09 14:50
(Received via mailing list)
I'm not sure what db you're using but if it's MySQL then it looks like
all db calls are bottlenecked down to just 3 methods:
	def command
	def read
	def write
in the mysql.rb file. So they would be pretty easy to trap; much easier
than wrapping all of your db calls, as you have pointed out.
5ceea097a3b29cb6a5da6705926410f4?d=identicon&s=25 gerret.apelt (Guest)
on 2005-12-09 16:41
(Received via mailing list)
Lou thats a good point and I am indeed using MySQL, so will give that
a shot and report back on whether it fixed the problem.

Would there be a reason why the adapter doesn't already do this
(reconnect when connection lost)? I can see that its not great for
performance, and there's no reason for the precaution when not
forking. But it doesn't feel right that using fork+AR would require
your hacking your DB adapter. Maybe an ActiveRecord::base option
'reconnect_on_lost_connect' would be the way to go.

At any rate thanks for the suggestion and I'll try it out later on.

cheers
Gerret
Ce60c4f78a63b0695e4dafc4bd7964f7?d=identicon&s=25 vanek (Guest)
on 2005-12-09 17:26
(Received via mailing list)
If the database is properly tuned then connections are automatically
recycled after X number of seconds. In MySQL, it's after wait_timeout
seconds. There really is no need to call an explicit 'close' on the
connection because MySQL does it automatically. I would hope that we
are given a config option in the future that precludes db connections
from explicitly being closed. That would solve this problem, and make
the code simpler.

As you point out, trapping for the lost connection is expensive, and
is probably why it is not being done.

I like the optional 'reconnect_on_lost_connect' idea.

I suspect that there's not a lot of people doing "fork"s with AR, which
is probably why this AR problem exists.
3ccecc71b9fb0a3d7f00a0bef6f0a63a?d=identicon&s=25 ksruby (Guest)
on 2005-12-09 18:15
(Received via mailing list)
I don't think this is AR related problem. You will experience the same
problem
just by using mysql driver alone. Everytime you fork a child ruby
process, it
inherits the same mysql connection from the parent and on exit it closes
it.

When I need to fork a background process, I create a special daemon
using Drb
library that accepts requests and processes them for me. So that I can
avoid
forking at all.

Kent.
24d2f8804e6bb4b7ea6bd11e0a586470?d=identicon&s=25 jeremy (Guest)
on 2005-12-09 20:52
(Received via mailing list)
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

> You write that the DB socket is closed when the forked process exits.
> So I'm guessing that is my problem.
>
> Any ideas from anybody are much appreciated -- I'm stumped. How can I
> do AR work in a forked process without borking my application?

Open a new connection in the forked child.

jeremy
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (Darwin)

iD8DBQFDmd9qAQHALep9HFYRAp0wAJ49xiLkudW6Rh23hN639SONCp8KLwCfVesF
krD7JqkztyWZXPqBuaxCP7M=
=nhjx
-----END PGP SIGNATURE-----
678fc3f0f17b58b60b73ff048fd67a7f?d=identicon&s=25 Francis Fish (fjfish)
on 2008-12-08 19:48
jeremy wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>> You write that the DB socket is closed when the forked process exits.
>> So I'm guessing that is my problem.
>>
>> Any ideas from anybody are much appreciated -- I'm stumped. How can I
>> do AR work in a forked process without borking my application?
>
> Open a new connection in the forked child.
>
> jeremy
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.2 (Darwin)
>
> iD8DBQFDmd9qAQHALep9HFYRAp0wAJ49xiLkudW6Rh23hN639SONCp8KLwCfVesF
> krD7JqkztyWZXPqBuaxCP7M=
> =nhjx
> -----END PGP SIGNATURE-----

Works for me

    ActiveRecord::Base.connection.reconnect!

Was having a problem with simple daemon and this fixed it.
D4036ffb76dbdf87588e09765ed66cad?d=identicon&s=25 Dan Michaelson (danmichaelson)
on 2010-04-03 20:04
Resurrecting an old but helpful thread just to note for future Googlers
that vanek's below solution works in a pinch (even if it might not be
the most efficient)-- except "if e.to_s =~/away/" doesn't catch all
possible exceptions that can raised by the db connection your forked
process closed upon exit. Every so often a forked process will exit in
the middle of an in-process db query in the parent, and you'll get a
still-uncaught exception, "Lost connection to MySQL server during
query". So you should at least do "if e.to_s =~ /has gone away/ or
e.to_s =~ /Lost connection/" to catch that one as well.


vanek wrote:
> somebody posted their solution to this problem a few days ago.
> wouldn't hurt to give it a shot:
>
>   def something
>     begin
>       fork do
>         emails.each { |email|
>           email.send  # calls SMTP
>           email.log      # calls ActiveRecord subclass
>         }
>       end
>
>       arInstance.do_something # do a bunch of AR stuff
>
>     rescue ActiveRecord::StatementInvalid => e
>       if e.to_s =~ /away/
>              ActiveRecord::Base.establish_connection and retry
>       else
>              raise e
>       end
>     end
>   end
84c94b8328b9978f952178d8dea44735?d=identicon&s=25 Ed Murray (emurray100)
on 2011-05-20 20:35
"The child process can exit using Kernel.exit! to avoid running any
at_exit functions."

This seems to have fixed the issue for me.  I have no at_exit functions
I need executed.
This topic is locked and can not be replied to.