Backgroundrb issues

Hi there,

I’m making a site that does a lot of image processing. In order to
avoid timeouts while the clients image is being processed I have used
backgroundrb to process the images.

However, I have a few small issues:

  1. Images tend to come in short, intense bursts. I want all the
    images to be processed at the same time, but it seems like drb is
    processing the images in sequence, with each image waiting for the
    next to be processed before it starts processing. Is there any way to
    make drb process them all concurrently?

  2. Sometimes my worker just doesn’t start. I can’t find any errors
    anywhere. But looking the log indicated that the worker never got the
    message to start. I can’t see any correlation between image size, type
    or anything. It seems pretty random but happens about once every 10
    images. If it was a problem with my code I would expect to see an
    error message somewhere, but I can’t find one. I am assuming that
    something goes wrong, or gets lost, and that the error is never
    logged.

Does anyone have any suggestions?

Thanks,
Jonzo.

Hi Jonzo,

I’m also using backgroundrb for images processing. Mine are processing
the images one at a time - not really an issue for my application,
after all the application has returned control to the user. I’m not
getting any problems with ‘bursts’ of activity or missing workers btw
I’m using the latest backgroundrb. Processing time seems very linear.
Deployment OS is Centos, RMagick for processing and Mysql for DB
( images stored in filesystem).

regards
Kevin

Thanks Kevin, it’s good to know that it does work for some people!

The bursts of activity isn’t actually a bug or problem, it’s just the
way my application behaves. Typically people will upload 10 or so
images at once so I am starting that many workers. It just seems like
sometimes the workers don’t start, and when one of the workers doesn’t
start there is nothing printed out in any log files. no errors, and
none of the log entries I have put in my code either.

Can you see any errors in my logic/code?

Here is my worker code:

============================================

class FileProcessorWorker < BackgrounDRb::MetaWorker
set_worker_name :file_processor_worker
def create(args = nil)
# this method is called, when worker is loaded for the first time
end

def process( file_id )
begin
logger.info “processing file now”
file = UploadedFile.find( file_id )
logger.info “file: #{ file.inspect }”
if file && file.original
file.process
logger.info “file processing completed”
else
logger.info “file not processed”
end
rescue
logger.info “file processing error: #{ $! }”
end
end

end

============================================

And I am starting my worker like this (after my model is created):

def start_worker
MiddleMan.worker(:file_processor_worker).process(id)
end

Thanks,
Jonzo.

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs