# Shared Queue / Exclusive Query Results **URL:** \ **Category:** Rails\ **Created:** [May 9, 2006, 6:49pm UTC](https://www.ruby-forum.com/t/shared-queue-exclusive-query-results/60044 "2006-05-09T18:49:22Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![dstocks](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/48db29/32.png) [@dstocks](https://www.ruby-forum.com/u/dstocks)\ **Post date:** [May 9, 2006, 6:49pm UTC](https://www.ruby-forum.com/t/shared-queue-exclusive-query-results/60044/1 "2006-05-09T18:49:22Z")
My application is going to have a work queue of scheduled tasks stored in the DB. This work queue will be shared across multiple Rails app servers. I need a way to ensure that no two servers get the same jobs if multiple servers pop jobs off the top of the queue simultaneously. Is there a fairly simple way to acheive this? Or do I need to come up with some fancy secondary server to dispatch and marshall (serialize) job assignments? Thanks! - Don ---
**Author:** ![dstocks](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/48db29/32.png) [@dstocks](https://www.ruby-forum.com/u/dstocks)\ **Post date:** [May 9, 2006, 8:21pm UTC](https://www.ruby-forum.com/t/shared-queue-exclusive-query-results/60044/2 "2006-05-09T20:21:11Z")
On May 9, 2006, at 9:49 AM, Don S. wrote: > My application is going to have a work queue of scheduled tasks stored > in the DB. This work queue will be shared across multiple Rails app > servers. I need a way to ensure that no two servers get the same jobs > if multiple servers pop jobs off the top of the queue simultaneously. > > Is there a fairly simple way to acheive this? Or do I need to come up > with some fancy secondary server to dispatch and marshall (serialize) > job assignments? Attempt to lock the job after popping it from the queue: class Job \< AR::Base #self.locking\_column = ‘lock\_version’ belongs\_to :queue end class Queue \< AR::Base has\_many :pending\_jobs, :class\_name =\> Job, :conditions =\> ‘pending=true’ ``` def pop if target = pending_jobs.find(:first) target.pending = false target.save! target end rescue ActiveRecord::StaleObjectError # Grab another if someone else locked ours first. retry end ``` end jeremy ---
**Author:** ![dstocks](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/48db29/32.png) [@dstocks](https://www.ruby-forum.com/u/dstocks)\ **Post date:** [May 9, 2006, 9:09pm UTC](https://www.ruby-forum.com/t/shared-queue-exclusive-query-results/60044/3 "2006-05-09T21:09:44Z")
Fantastic input Jeremy. Thanks! That really solves a lot of problems. I’ll have to experiment and give it a test drive. If it works well in a very busy environment then I’ll be set. Thanks a ton! By the way, there may be a lot of jobs and many servers accessing the queue. What are your thoughts on the most efficient way to fetch multiple jobs at a time? - Don ---
**Author:** ![dstocks](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/48db29/32.png) [@dstocks](https://www.ruby-forum.com/u/dstocks)\ **Post date:** [May 9, 2006, 10:23pm UTC](https://www.ruby-forum.com/t/shared-queue-exclusive-query-results/60044/4 "2006-05-09T22:23:03Z")
Matt Bauer wrote: > I’m not sure about your application but maybe you should look into > Distributed Ruby and/or Rinda. It provides a lot of functionality for > working with queues and tasks in a distributed space. > > Matt I looked into and experimented with Rinda and DRb but I couldn’t figure out how it would solve my original problem without introducing a single point of failure. If all my Rails servers used the same intance of my Queue via DRb, then I would need to engineer a contignecy for failover. And that would probably mean multiple DRb servers, in which case I’d need to do what Jeremy suggests (or something like it) anyway. Additionaly, since my queue is comprised of fairly dynamic ActiveRecord objects, they go out of sync with the locally cached objects published via DRb. So the DRb server would need to continually refresh the objects in its cache from the DB. Although Jeremy’s solution is less sexy than a Distributed Ruby design, it seems like it’s less complex. I’m brand new to Ruby / Rails / DRb / Rinda. So I’m probably missing a lot here. Any insight is always appreciated! Thanks, Don ---
**Author:** ![dstocks](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/48db29/32.png) [@dstocks](https://www.ruby-forum.com/u/dstocks)\ **Post date:** [May 9, 2006, 9:20pm UTC](https://www.ruby-forum.com/t/shared-queue-exclusive-query-results/60044/5 "2006-05-09T21:20:08Z")
I’m not sure about your application but maybe you should look into Distributed Ruby and/or Rinda. It provides a lot of functionality for working with queues and tasks in a distributed space. Matt ---
**Author:** ![dstocks](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/48db29/32.png) [@dstocks](https://www.ruby-forum.com/u/dstocks)\ **Post date:** [May 10, 2006, 5:49pm UTC](https://www.ruby-forum.com/t/shared-queue-exclusive-query-results/60044/6 "2006-05-10T17:49:21Z")
On May 9, 2006, at 12:09 PM, Don S. wrote: > By the way, there may be a lot of jobs and many servers accessing the > queue. What are your thoughts on the most efficient way to fetch > multiple jobs at a time? I’d add a :lock option to AR::Base#find so you could: class Job \< AR::Base belongs\_to :owner ``` def self.claim!(owner, n = 10) # select * from jobs where owner_id is null limit 10 for update jobs = find(:all, :conditions => 'owner_id is null', :limit => ``` n, :lock =\> ‘for update’) ``` # update jobs set owner_id=1 where id in (1,2,3,4,5) update_all(['owner_id=?', owner], ['id in (?)', jobs.map { |j| ``` j.id }]) ``` jobs end ``` end select … for update gives you an exclusive lock on the selected rows so others can’t set themselves as owner before you do. Best, jeremy