Forum: Ruby on Rails Multiple workling instances

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
A3b04ae63775ecf10247321bec678fe9?d=identicon&s=25 Rob W. (rfw21)
on 2009-03-25 11:37
Hi,
I'm running Starling and Workling to process background jobs. I've four
instances of starling running on the same server, and I'm trying to get
Workling to use them all. However, all of the jobs seem to be queued up
and carried out one by one, instead of up to four being process
simultaneously.

As a result, I've got a very long job queue which is taking a long time
to process, and I'd like to reduce it.

I've set :multiple to true in script/workling_client.rb, but this
doesn't seem to help.

I'm very inexperienced with both Starling and Workling, so it's quite
possible I'm either missing something simple, or approaching this the
wrong way entirely...

Any suggestions or insight much appreciated!

best
0ce41a1e93464552598f291538fe26b8?d=identicon&s=25 Mark Lamp (lamplighter)
on 2009-03-29 21:52
(Received via mailing list)
I actually just was in the same situation.  I also had thought
multiple Starling instances would help things, but it didn't.  The
bottleneck was Workling, and I imagine it is the same for you.
Despite the lack of documentation, it turns out to be extremely easy
to run several instances:

1) In script/workling_client.rb, change :multiple from false to true
as you already have, so the Daemon knows to allow for multiple
instances
2) Just start more instances by calling "script/workling_client start"
several times.  It will create workling0.pid, workling1.pid, etc. I'm
not sure how to properly setup god.rb to monitor the multiple
instances so if you do please let me know :)

FYI, you could also create multiple worker classes (my_worker.rb,
another_worker.rb etc), and each Workling instance would use a thread
for each class.  So if you split up your methods into several classes,
it might help things even more.
0ce41a1e93464552598f291538fe26b8?d=identicon&s=25 Mark Lamp (lamplighter)
on 2009-03-29 21:58
(Received via mailing list)
PS.  Here is an extremely helpful series of blog posts for starling
and workling:
http://davedupre.com/2008/03/25/ruby-background-ta...
Cff79add519292917eb4660e82a78f06?d=identicon&s=25 Mirko Froehlich (Guest)
on 2009-04-11 01:16
(Received via mailing list)
I have modified my workling_client script to better handle multiple
worklings. This approach gives you full control over the workling
process names and pid files, which you can then use to cleanly manage
the worklings using God or Monit.

See below for my (admittedly hacky) solution:

#!/usr/bin/env ruby
require 'rubygems'
require 'daemons'

# This is a bit hacky, but we don't have a clean way to hook into the
parsed
# options, as Daemons is managing these internally.
number = 0
for i in 0..ARGV.length
  if ARGV[i] == '--number'
    number = ARGV[i + 1]
    2.times {ARGV.delete_at(i)}
  end
end

puts "Workling Number: #{number}"

workling = File.join(File.dirname(__FILE__), '..', 'vendor',
'plugins', 'workling', 'script', 'listen.rb')
options = {
  :app_name   => "workling-#{number}-client",
  :ARGV       => ARGV,
  :dir_mode   => :normal,
  :dir        => File.join(File.dirname(__FILE__), '..', 'log'),
  :log_output => true,
  :multiple   => false,
  :backtrace  => true,
  :monitor    => false
}

Daemons.run(workling, options)
This topic is locked and can not be replied to.