Forum: Ruby on Rails ActiveRecord on Threads: close and reopen the connection?

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
74e21b54f21ba89e663180c3513fe155?d=identicon&s=25 Peter Bengtson (peter)
on 2007-01-31 19:26
(Received via mailing list)
I'm writing a Rails application which does not have a web frontend
(no controllers and no views, only models). Rather, it is a faceless
server, supporting many tens of thousands of threads and socket
connections. Some of these are fairly short-lived, perhaps a second
or two at the most, but the majority of them live longer, possibly up
to several hours or more.

Writing a server application in Rails has its advantages - among them
is the testing framework, and of course the abstractions provided by
ActiveRecord.

However, it is impractical to allot a database connection per thread,
since there are so many of them. And most of the work done in a
thread involves manipulation of ActiveRecord objects only initially,
at the start of a thread's life, and perhaps a couple of times
thereafter.

For this application, it would make sense to close the db connection
and reopen it when needed.

My question to those of you who know about the inner workings of
ActiveRecord is: what is the best way of disconnecting from the
database? Of course, I can wrap all ActiveRecord accesses in blocks,
like this:

  with_connection do { MyModel.find(:all).each ... }

where with_connection could be defined as

  def with_connection
    begin
      ActiveRecord::Base.establish_connection
      result = yield
    ensure
      ActiveRecord::Base.remove_connection
    end
    result
  end

With_connection could of course also maintain a pool of connections.

But is there a configuration alternative that automatically makes
ActiveRecord drop the connection after each query?

I should add that ActiveRecord::Base.allow_concurrency = true, and
that the db is Postgres, with concurrency enabled in the database.yml
file.

  / Peter Bengtson
6076c22b65b36f5d75c30bdcfb2fda85?d=identicon&s=25 Ezra Zygmuntowicz (Guest)
on 2007-01-31 21:51
(Received via mailing list)
Hi~

On Jan 31, 2007, at 10:24 AM, Peter Bengtson wrote:

> ActiveRecord.
> My question to those of you who know about the inner workings of
>       ActiveRecord::Base.establish_connection
> ActiveRecord drop the connection after each query?
>
> I should add that ActiveRecord::Base.allow_concurrency = true, and
> that the db is Postgres, with concurrency enabled in the database.yml
> file.
>
>   / Peter Bengtson


Peter-

  The above could work but in my experiments with using AR in threaded
mode the easiest way to make sure connections close is to
verify_active_connections. I assume you have a server.accept loop
that is accepting new connections and firing up a thread for each
one? AR will make a connection for each thread. But if you put a
counter around your server.accept loop and then
verfiy_active_connections! every 20-50 times through the loop it will
reap old connections and keep things fairly clean.

hits = 0
while server.accept
   hits += 1
   Thread.new do
       # do your thang in here that calls AR
   end
   ActiveRecord::Base.verify_active_connections! if hits % 30
end

  That seems to work well when allow_concurrency is set to true. But
in your case of needing thousands of sockets open then you may want
to look at an even driven server like EvenMachine on rubyforge.


Cheers-
-- Ezra Zygmuntowicz
-- Lead Rails Evangelist
-- ez@engineyard.com
-- Engine Yard, Serious Rails Hosting
-- (866) 518-YARD (9273)
This topic is locked and can not be replied to.