Stop browser timeout with large file uploads + processing?


#1

Hi

Im uploading a file to my rails app, i’s usually a large file, 5-50mb.
Once the file has been uploaded it must be parsed and processed, this
can take a very long time, 1-20mins. The problem is that most of the
time my browser timeouts.

Im not really sure how to solve it. One idea is that i have a script
run constantly on the server that checks for uploaded files in a
specified folder. This would prevent timeouts. The user would simply
upload the file and be shown a screen telling them that the data will be
in the system in 1-20 mins.

Im not sure if this is an ideal solution though. What are your
thoughts? Does anyone have experience with this or have better
solution?


#2

Chris R. wrote:

Hi

Im uploading a file to my rails app, i’s usually a large file, 5-50mb.
Once the file has been uploaded it must be parsed and processed, this
can take a very long time, 1-20mins. The problem is that most of the
time my browser timeouts.

Im not really sure how to solve it. One idea is that i have a script
run constantly on the server that checks for uploaded files in a
specified folder. This would prevent timeouts. The user would simply
upload the file and be shown a screen telling them that the data will be
in the system in 1-20 mins.

Im not sure if this is an ideal solution though. What are your
thoughts? Does anyone have experience with this or have better
solution?

After the browser uploads the file trigger a background process with an
ajax call. This then at various points in this long running process
update a session variable with the progress on your long action:

def long_action
#do stuff
session[:progress] = 0.1
session.update

#do stuff
session[:progress] = 0.2
session.update

end

Then have a periodic AJAX call to retrieve a progress bar partial based
on the session[:progress]

Progress: <%= (session[:progress] * 100).to_i %>%

I have used this technique with an action that uploads some large chunks
of data to external ftp servers, runs about 10-30 minutes. The ajax
keeps the browser from getting bored. Note this will not work in
webrick. It require a minimum of 2 server processes and Webrick only
uses 1. You need one proc to do the long running action, and one proc
to respond to the ajax queries.

You might even be able to do the same thing with the uploading of the
file itself as well, but I haven’t actually pulled that one off. Check
out the file_upload_progress plugin (i think thats whats its called) for
more info on ajaxy uploads.


#3

Ezra~

That log tailer looks amazing! I can’t wait!

~ Ben

On 5/5/06, Ezra Z. removed_email_address@domain.invalid wrote:

can take a very long time, 1-20mins. The problem is that most of the
Im not sure if this is an ideal solution though. What are your
session[:progress] = 0.1

You might even be able to do the same thing with the uploading of the
blog. Stay tuned for the released code which is forthcoming RSN.
-Ezra


Rails mailing list
removed_email_address@domain.invalid
http://lists.rubyonrails.org/mailman/listinfo/rails


Ben R.
303-947-0446
http://www.benr75.com


#4

Hi~

On May 5, 2006, at 10:01 AM, Alex W. wrote:

run constantly on the server that checks for uploaded files in a
After the browser uploads the file trigger a background process
session[:progress] = 0.2
chunks
called) for
more info on ajaxy uploads.

I have a plugin that will be released in the next few days that is

precisely written to accomodate this kind of long running background
process. It will even work with just one webrick or fcgi or whatever.
You can see some screencasts and the thoughts behind it here on my
blog. Stay tuned for the released code which is forthcoming RSN.

http://brainspl.at/articles/2006/05/04/preview-of-drbworker-plugin-
with-ajax-progress-bars

Cheers-
-Ezra