# backgroundRB problem (newbie)

**URL:** <https://www.ruby-forum.com/t/backgroundrb-problem-newbie/145732>\
**Category:** Rails\
**Created:** [August 13, 2008, 12:52pm UTC](https://www.ruby-forum.com/t/backgroundrb-problem-newbie/145732 "2008-08-13T12:52:15Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Iain\_A](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/6bbea6/32.png) [@Iain\_A](https://www.ruby-forum.com/u/Iain_A)\
**Post date:** [August 13, 2008, 12:52pm UTC](https://www.ruby-forum.com/t/backgroundrb-problem-newbie/145732/1 "2008-08-13T12:52:15Z")

</div>

Hello,

I am trying to upload and process very large CSV files. For this I am  
trying to use backgroundRB. The problem is that I cannot access a  
variable inside my worker until the methods have finished even though  
I am calling the method asynchronously.

My controller:

def authenticate\_import  
@dataset = Dataset.new params[:dataset]  
@dataset.campaign = @campaign  
#Check the normal info is valid  
if !@dataset.valid?  
render :update do |page|  
page.replace\_html ‘import’, :action =\> ‘import’  
end  
else  
@key = Time.now.to\_i  
@worker = MiddleMan.worker(:dataset\_worker, @key)  
@worker.async\_process\_csv(:arg =\> @dataset)  
render :update do |page|  
page.replace\_html ‘progress’, :partial =\>  
‘progress\_meter’, :object =\> @key, :locals =\> {:progress =\>  
@worker.progress}  
end  
end  
end

This should return the variable progress to the partial which at the  
moment simply displays the local progress.

My worker:

class DatasetWorker \< BackgrounDRb::MetaWorker  
set\_worker\_name :dataset\_worker

attr\_accessor :progress

def create(args = nil)  
logger.info “Dataset Worker setup”  
@progress = 0  
end

def process\_csv(dataset)  
logger.info “Process CSV method called”  
logger.info “Dataset is a #{dataset.type}”

```
logger.info "Dataset: begin parsing file"
  dataset.process_file
  @progress = 30
logger.info "Dataset: parsing complete"
logger.info "Dataset: Creating table in db"
  if dataset.save
    table = dataset.table
    total = table.size
        begin
            #Create the fields first, otherwise we would have to

```

check whether they had been created  
#for every row were inserting (alot of SQL calls for  
large CSV files)  
row\_cnt = 0  
table.headers.each do |field|  
DatasetField.create({:dataset =\>  
dataset, :dataset\_field\_name =\> field})  
end  
#reload fields  
dataset.dataset\_fields(true)  
table.by\_row!.each do |row|  
dataset.create\_row row  
row\_cnt++  
@progress = ((row\_cnt/total)\*70)+30  
end  
end  
end  
logger.info “Dataset: Finished creating table in db”  
end

end

I am trying to access the variable progress so that I can display it.  
Then I can periodically update the progress by using the job key. The  
problem is the Ajax requests doesn’t finish until the method  
process\_csv has finished.

Any help would be greatly appreciated

---

<div class="post-metadata">

**Author:** ![Iain\_A](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/6bbea6/32.png) [@Iain\_A](https://www.ruby-forum.com/u/Iain_A)\
**Post date:** [August 13, 2008, 4:08pm UTC](https://www.ruby-forum.com/t/backgroundrb-problem-newbie/145732/2 "2008-08-13T16:08:10Z")

</div>

On Wed, Aug 13, 2008 at 4:21 PM, Iain A.  
[removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> Hello,
> 
> I am trying to upload and process very large CSV files. For this I am  
> trying to use backgroundRB. The problem is that I cannot access a  
> variable inside my worker until the methods have finished even though  
> I am calling the method asynchronously.

Problem is by default worker is synchronous and uses event driven  
network programming, so next event is not picked until existing  
handler returns. But, if you #defer process\_csv inside a thread, next  
call to @worker.progress should return as you expect (for using #defer  
method inside worker, refer [backgroundrb.rubyforge.org](http://backgroundrb.rubyforge.org)).

Also, if you are going to use threads, then you need to protect  
@progress accessor with a mutex or something, so as concurrent  
threads, update it synchronously.

Another alternative is to use, bakcgroundrb with memcache for caching  
result objects (such as progress bar and stuff). From worker just do:

cache[:progress\_percent] = 80

and from rails:

MiddleMan.worker(:foo\_worker).ask\_result(:progress\_percent)

(refer official docs for more info).

> ```
> else
> 
> ```
> 
> def create(args = nil)  
> @progress = 30  
> row\_cnt = 0  
> end  
> process\_csv has finished.
> 
> Any help would be greatly appreciated
> 
> >

–  
Let them talk of their oriental summer climes of everlasting  
conservatories; give me the privilege of making my own summer with my  
own coals.

[http://gnufied.org](http://gnufied.org)