# Offloading big uploads to front-end webserver

**URL:** <https://www.ruby-forum.com/t/offloading-big-uploads-to-front-end-webserver/132936>\
**Category:** Rails\
**Created:** [March 25, 2008, 10:46pm UTC](https://www.ruby-forum.com/t/offloading-big-uploads-to-front-end-webserver/132936 "2008-03-25T22:46:36Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![casper\_the\_ghost](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/58956e/32.png) [@casper\_the\_ghost](https://www.ruby-forum.com/u/casper_the_ghost)\
**Post date:** [March 25, 2008, 10:46pm UTC](https://www.ruby-forum.com/t/offloading-big-uploads-to-front-end-webserver/132936/1 "2008-03-25T22:46:36Z")

</div>

I’m doing a website which has quite a few big uploads / downloads to  
manage.

Since rails isn’t multithreaded (yet), this means every big upload/  
download will keep a mongrel/thin/whatever thread for itself doing  
nothing but waiting for the transfer to finish.

Now, for public files, this of course gets handled by my frontend  
nginx server.

For private files, I use the X-Accel-Redirect header, so the transfer  
gets offloaded to nginx as well.

Only for uploads, I couldn’t find a nice way to handle them.  
Is there a way to somehow tell nginx to first receive the whole file,  
then transfer it to mongrel/thin so it doesn’t lock up a full  
instance?  
Any way to do this, while first consulting rails if this should be  
allowed?

Any help is greatly appreciated.

Greetings,  
Mathijs

---

<div class="post-metadata">

**Author:** ![casper\_the\_ghost](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/58956e/32.png) [@casper\_the\_ghost](https://www.ruby-forum.com/u/casper_the_ghost)\
**Post date:** [March 26, 2008, 12:35am UTC](https://www.ruby-forum.com/t/offloading-big-uploads-to-front-end-webserver/132936/2 "2008-03-26T00:35:59Z")

</div>

On 25 Mar 2008, at 22:45, removed\_email\_address@domain.invalid wrote:

> For private files, I use the X-Accel-Redirect header, so the transfer  
> gets offloaded to nginx as well.
> 
> Only for uploads, I couldn’t find a nice way to handle them.  
> Is there a way to somehow tell nginx to first receive the whole file,  
> then transfer it to mongrel/thin so it doesn’t lock up a full  
> instance?  
> Any way to do this, while first consulting rails if this should be  
> allowed?

First of all, Mongrel is multithreaded and Mongrel doesn’t hand off  
the uploaded file to Rails until after the upload itself (i.e. the  
upload itself doesn’t block the mongrel at all). That said, after the  
upload the Mongrel will be locked for the duration of processing in  
Rails.

Depending on what you are actually doing in Rails, you have several  
possibilities:

1. Just saving the file to disk: just let Rails handle it using a  
Tempfile, see the attachment\_fu source code. It’s pretty fast and  
won’t lock your Mongrel for long since the Rails request, which IS  
single threaded, won’t take long
2. Extensive processing: offload the upload to a background process,  
and here you have another choice:
  - BackgroundRB (and other background processors): has access to  
the Rails stack and runs behind Rails, i.e. Rails will hand over the  
file to the background worker
  - Merb: a Rails like framework that contrary to Rails is  
multithreaded, it can sit behind a single mongrel and process heaps of  
uploads without locking the mongrel. You can plugin ActiveRecord into  
merb for model access. You’ll also need some rewrite magic to direct  
uploads to the Merb application if you want the rest served through  
Rails.

Best regards

Peter De Berdt