Forum: Ruby on Rails Shared Queue / Exclusive Query Results

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
Don S. (Guest)
on 2006-05-09 20:49
My application is going to have a work queue of scheduled tasks stored
in the DB.  This work queue will be shared across multiple Rails app
servers.  I need a way to ensure that no two servers get the same jobs
if multiple servers pop jobs off the top of the queue simultaneously.

Is there a fairly simple way to acheive this?  Or do I need to come up
with some fancy secondary server to dispatch and marshall (serialize)
job assignments?

Thanks! - Don
Jeremy K. (Guest)
on 2006-05-09 22:21
(Received via mailing list)
On May 9, 2006, at 9:49 AM, Don S. wrote:
> My application is going to have a work queue of scheduled tasks stored
> in the DB.  This work queue will be shared across multiple Rails app
> servers.  I need a way to ensure that no two servers get the same jobs
> if multiple servers pop jobs off the top of the queue simultaneously.
>
> Is there a fairly simple way to acheive this?  Or do I need to come up
> with some fancy secondary server to dispatch and marshall (serialize)
> job assignments?

Attempt to lock the job after popping it from the queue:

   class Job < AR::Base
     #self.locking_column = 'lock_version'
     belongs_to :queue
   end

   class Queue < AR::Base
     has_many :pending_jobs, :class_name => Job, :conditions =>
'pending=true'

     def pop
       if target = pending_jobs.find(:first)
         target.pending = false
         target.save!
         target
       end
     rescue ActiveRecord::StaleObjectError
       # Grab another if someone else locked ours first.
       retry
     end
   end

jeremy
Don S. (Guest)
on 2006-05-09 23:09
Fantastic input Jeremy.  Thanks!  That really solves a lot of problems.
I'll have to experiment and give it a test drive.  If it works well in a
very busy environment then I'll be set.  Thanks a ton!

By the way, there may be a lot of jobs and many servers accessing the
queue.  What are your thoughts on the most efficient way to fetch
multiple jobs at a time?

 - Don
Matt B. (Guest)
on 2006-05-09 23:20
I'm not sure about your application but maybe you should look into
Distributed Ruby and/or Rinda.  It provides a lot of functionality for
working with queues and tasks in a distributed space.

Matt
Don S. (Guest)
on 2006-05-10 00:23
Matt Bauer wrote:
> I'm not sure about your application but maybe you should look into
> Distributed Ruby and/or Rinda.  It provides a lot of functionality for
> working with queues and tasks in a distributed space.
>
> Matt

I looked into and experimented with Rinda and DRb but I couldn't figure
out how it would solve my original problem without introducing a single
point of failure. If all my Rails servers used the same intance of my
Queue via DRb, then I would need to engineer a contignecy for failover.
And that would probably mean multiple DRb servers, in which case I'd
need to do what Jeremy suggests (or something like it) anyway.
Additionaly, since my queue is comprised of fairly dynamic ActiveRecord
objects, they go out of sync with the locally cached objects published
via DRb.  So the DRb server would need to continually refresh the
objects in its cache from the DB.

Although Jeremy's solution is less sexy than a Distributed Ruby design,
it seems like it's less complex.

I'm brand new to Ruby / Rails / DRb / Rinda.  So I'm probably missing a
lot here.  Any insight is always appreciated!

Thanks, Don
Jeremy K. (Guest)
on 2006-05-10 19:49
(Received via mailing list)
On May 9, 2006, at 12:09 PM, Don S. wrote:
> By the way, there may be a lot of jobs and many servers accessing the
> queue.  What are your thoughts on the most efficient way to fetch
> multiple jobs at a time?

I'd add a :lock option to AR::Base#find so you could:

   class Job < AR::Base
     belongs_to :owner

     def self.claim!(owner, n = 10)
       # select * from jobs where owner_id is null limit 10 for update
       jobs = find(:all, :conditions => 'owner_id is null', :limit =>
n, :lock => 'for update')

       # update jobs set owner_id=1 where id in (1,2,3,4,5)
       update_all(['owner_id=?', owner], ['id in (?)', jobs.map { |j|
j.id }])

       jobs
     end
   end

select ... for update gives you an exclusive lock on the selected
rows so others can't set themselves as owner before you do.

Best,
jeremy
This topic is locked and can not be replied to.