ActiveRecord: using with multiple threads/DB connections


#1

Hi,

I’m totally new to Ruby and Rails and I’m trying to set up a prototype
web site. Unfortunately, I’m having problems with ActiveRecord using
multiple threads.

What I want to do is fairly simple. I have a Controller that receives
and saves data from the web in the DB, but once that’s done I want to
process this data in another thread and then save the result in some
other DB table. I want to do the processing in another thread because it
may take well over a minute.

When I do this, it works fine as long as I’m not hitting the web server
with additional requests while it is processing. Otherwise, I get all
sorts of errors either DB-related (StatementInvalid) or in rails itself
(“nil received when it was not expected”) or WEBrick dumps a core.

BTW I’m using MySQL with InnoDB tables. I believe it should work fine
with default settings, but maybe I’m missing something there. I don’t
think so though.

I read all the ActiveRecord doc I could find and I believe the problem
is that ActiveRecord globally maps a class to a connection. So there is
no way to use a separate DB connection to do my asynchronous processing.
Since all DB statements (across multiple threads) are executed in the
same connections there is bound to be trouble.

I copy the Controller and some errors I’ve seen below. You can see what
I mean directly in code.

This seems like a very simple case, I’m sure there is something I’m
doing wrong. What is the ‘good’ way to do this? Having a separate
process to do the backend processing? But then, what about multiple
threads using ActiveRecord in this backend process?

What am I doing wrong/missing?

I went over the doc many times and couldn’t find anything to solve my
problem (I checked ActiveRecord::Base.connection and
.establish_connection, but it doesn’t seem like it will help me in this
case.) I read ActiveRecord is thread safe, but that’s not what I’m
experiencing. Where does the synchronization take place?

Again, I’m new to RoR so that may be stupid questions. Feel free to send
me a RTFM, but please provide a link to said manual! :wink:

Thanks,

Max

class ArticlesController < ApplicationController

def create
@article = Article.new(params[:article])
if processNewArticle(@article)
flash[:notice] = ‘Article was successfully created.’
redirect_to :action => ‘list’
else
render :action => ‘new’
end
end

def processNewArticle(article)
…do some validation…
if (!article.save)
false
end
# at this point, we know everything is ok
# start processing
Thread.new {
… some very long processing …
}
true
end


ActiveRecord::StatementInvalid in Articles#new

Mysql::Error: Lost connection to MySQL server during query: SHOW FIELDS
FROM articles

RAILS_ROOT: script/…/config/…
Application Trace | Framework Trace | Full Trace

/usr/lib/ruby/gems/1.8/gems/activerecord-1.13.2/lib/active_record/connection_adapters/abstract_adapter.rb:88:in
log' /usr/lib/ruby/gems/1.8/gems/activerecord-1.13.2/lib/active_record/connection_adapters/mysql_adapter.rb:180:inexecute’
/usr/lib/ruby/gems/1.8/gems/activerecord-1.13.2/lib/active_record/connection_adapters/mysql_adapter.rb:278:in
columns' /usr/lib/ruby/gems/1.8/gems/activerecord-1.13.2/lib/active_record/base.rb:734:incolumns’
/usr/lib/ruby/gems/1.8/gems/activerecord-1.13.2/lib/active_record/base.rb:1663:in
attributes_from_column_definition' /usr/lib/ruby/gems/1.8/gems/activerecord-1.13.2/lib/active_record/base.rb:1185:ininitialize_without_callbacks’
/usr/lib/ruby/gems/1.8/gems/activerecord-1.13.2/lib/active_record/callbacks.rb:236:in
initialize' ./script/../config/../app/controllers/articles_controller.rb:19:innew’
./script/…/config/…/app/controllers/articles_controller.rb:19:in `new’


Mysql::Error in Articles#list

Lost connection to MySQL server during query

RAILS_ROOT: script/…/config/…
Application Trace | Framework Trace | Full Trace

/usr/lib/ruby/gems/1.8/gems/activerecord-1.13.2/lib/active_record/vendor/mysql.rb:1093:in
read' /usr/lib/ruby/gems/1.8/gems/activerecord-1.13.2/lib/active_record/vendor/mysql.rb:500:inread’
/usr/lib/ruby/gems/1.8/gems/activerecord-1.13.2/lib/active_record/vendor/mysql.rb:151:in
real_connect' /usr/lib/ruby/gems/1.8/gems/activerecord-1.13.2/lib/active_record/connection_adapters/mysql_adapter.rb:316:inconnect’
/usr/lib/ruby/gems/1.8/gems/activerecord-1.13.2/lib/active_record/connection_adapters/mysql_adapter.rb:164:in
reconnect!' /usr/lib/ruby/gems/1.8/gems/activerecord-1.13.2/lib/active_record/connection_adapters/abstract/connection_specification.rb:103:inretrieve_connection’
/usr/lib/ruby/gems/1.8/gems/activerecord-1.13.2/lib/active_record/connection_adapters/abstract/connection_specification.rb:20:in
connection' /usr/lib/ruby/gems/1.8/gems/activerecord-1.13.2/lib/active_record/base.rb:518:incount_by_sql’
/usr/lib/ruby/gems/1.8/gems/activerecord-1.13.2/lib/active_record/base.rb:511:in
count' /usr/lib/ruby/gems/1.8/gems/actionpack-1.11.2/lib/action_controller/pagination.rb:167:incount_collection_for_pagination’
/usr/lib/ruby/gems/1.8/gems/actionpack-1.11.2/lib/action_controller/pagination.rb:188:in
paginator_and_collection_for' /usr/lib/ruby/gems/1.8/gems/actionpack-1.11.2/lib/action_controller/pagination.rb:124:inpaginate’
./script/…/config/…/app/controllers/articles_controller.rb:11:in
`list’


#2

Digging through the code, I just saw that it is WEBrick that disables
concurrency…

I now manually set allow_concurrency (to true) in my models. Don’t know
if this is kosher, but it will suffice for now.