I’m not doing anything fancy. On an action I simply create the worker
def test
MiddleMan.new_worker(:class => :test_worker)
end
and in the worker I just write to the log. It never gets that far
though, because when it tries to create the worker it keeps raising the
failed to find slave socket error. Has anyone run into something like
this before, or have any ideas what could be causing the problem? I’m
completely stumped.
Version Info
backgroundrb (0.2.1)
slave (1.2.1)
daemons (1.0.6)
We are having the same problem… For our case, we have noticed that
the /tmp/backgroundrb.$$ (i.e. /tmp/backgroundrb.26943/ where 26943
is the PID) gets filled up w/ worker socket files which don’t get
cleaned up even after the worker has stopped. After 42 of those files
are created, background simply stops processing, period. It had to be
restarted. We upgraded to backgroundrb from the Dejavu trunk, and it
helped in that when this issue happened, backgroundrb would at least
try again. It still gets stuck, but I can get it to start processing
again if I remove the workder files from /tmp/backgroundrb.$$/. This
42 file max happens for all workers.
Backgroundrb seems to function very well when it comes to ‘triggered’
jobs. The time scheduling though is very choppy.We converted the jobs
to a custom daemon w/ thread calls. This was helpful example code for
daemons in case you need it: http://snippets.dzone.com/posts/show/2265
Yea, we did the self.delete. You are running 120,000 time-triggered
workers with no problems? Standard triggered background jobs works
fine, the time scheduled ones do the behavior described.
thanks
On Sep 5, 8:26 am, Roderick van Domburg <rails-mailing-l…@andreas-
We are having the same problem… For our case, we have noticed that
the /tmp/backgroundrb.$$ (i.e. /tmp/backgroundrb.26943/ where 26943
is the PID) gets filled up w/ worker socket files which don’t get
cleaned up even after the worker has stopped. After 42 of those files
are created, background simply stops processing, period. It had to be
restarted. We upgraded to backgroundrb from the Dejavu trunk, and it
helped in that when this issue happened, backgroundrb would at least
try again. It still gets stuck, but I can get it to start processing
again if I remove the workder files from /tmp/backgroundrb.$$/. This
42 file max happens for all workers.
Do you call self.delete at the end of each worker to remove it from the
queue? It’s also good practice to wrap the entire do_work call in a
begin…rescue statement to ensure that the self.delete call will get
called.
We’re running several large-scale deployments of BackgrounDRb that are
iterating over 120,000 workers and are loving it.
Ooh ok, thanks. So that’s the difference. In essence your using
background as a daemon. We’ve been trying to use it with multiple
entries to run time-scheduled jobs and having the behavior mentioned.
We reverted to using a daemon threads for jobs and our own methods of
scheduling.
Yea, we did the self.delete. You are running 120,000 time-triggered
workers with no problems? Standard triggered background jobs works
fine, the time scheduled ones do the behavior described.
We schedule a single “maintenance worker” that in turn spawns any other
workers as necessary depending on the current time and the tasks
required. So our backgroundrb_schedule.yml only contains a single entry.
