# Uploading Large Files

**URL:** <https://www.ruby-forum.com/t/uploading-large-files/133470>\
**Category:** Rails\
**Created:** [March 31, 2008, 9:24pm UTC](https://www.ruby-forum.com/t/uploading-large-files/133470 "2008-03-31T21:24:51Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![jimmypalmer](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/j/bbe5ce/32.png) [@jimmypalmer](https://www.ruby-forum.com/u/jimmypalmer)\
**Post date:** [March 31, 2008, 9:24pm UTC](https://www.ruby-forum.com/t/uploading-large-files/133470/1 "2008-03-31T21:24:51Z")

</div>

I built an app that allows users to purchase video files to download,  
but unfortunately media temple’s grid server isn’t letting me upload  
large files. Everything goes great unless the files are over 100 mb.  
I’m using attachement\_fu.

Is there a service or technique that can allow me to put large files on  
my server.

---

<div class="post-metadata">

**Author:** ![jimmypalmer](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/j/bbe5ce/32.png) [@jimmypalmer](https://www.ruby-forum.com/u/jimmypalmer)\
**Post date:** [March 31, 2008, 10:32pm UTC](https://www.ruby-forum.com/t/uploading-large-files/133470/2 "2008-03-31T22:32:07Z")

</div>

Sounds like a case for Amazon S3.

On Mar 31, 3:24 pm, Jimmy P. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)

---

<div class="post-metadata">

**Author:** ![jimmypalmer](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/j/bbe5ce/32.png) [@jimmypalmer](https://www.ruby-forum.com/u/jimmypalmer)\
**Post date:** [March 31, 2008, 10:34pm UTC](https://www.ruby-forum.com/t/uploading-large-files/133470/3 "2008-03-31T22:34:28Z")

</div>

On 31 Mar 2008, at 21:24, Jimmy P. wrote:

> I built an app that allows users to purchase video files to download,  
> but unfortunately media temple’s grid server isn’t letting me upload  
> large files. Everything goes great unless the files are over 100 mb.  
> I’m using attachement\_fu.
> 
> Is there a service or technique that can allow me to put large files  
> on  
> my server.

FTP them, then use the Ruby File classes to read them into your  
attachment\_fu models. But my worry is how you’re going to serve those  
files if mediatemple won’t install the xsendfile module. If you’re  
going to stream those using send\_file or send\_data, you’re going to  
get into some serious problems, no matter how they serve their rails  
applications. I’d advise you to look into Amazon S3 since you seem to  
be charging for your videos anyway?

Best regards

Peter De Berdt

---

<div class="post-metadata">

**Author:** ![jimmypalmer](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/j/bbe5ce/32.png) [@jimmypalmer](https://www.ruby-forum.com/u/jimmypalmer)\
**Post date:** [June 10, 2008, 2:03am UTC](https://www.ruby-forum.com/t/uploading-large-files/133470/4 "2008-06-10T02:03:18Z")

</div>

Hey Peter - I thought I would just send you a direct message. You  
posted some advise on this thread for me a couple months back and was  
wondering if you have any further advise for me. See latest post  
below.

thanks

On Jun 9, 6:59 pm, Jimmy P. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)

---

<div class="post-metadata">

**Author:** ![jimmypalmer](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/j/bbe5ce/32.png) [@jimmypalmer](https://www.ruby-forum.com/u/jimmypalmer)\
**Post date:** [June 10, 2008, 1:59am UTC](https://www.ruby-forum.com/t/uploading-large-files/133470/5 "2008-06-10T01:59:57Z")

</div>

> FTP them, then use the Ruby File classes to read them into your  
> attachment\_fu models. But my worry is how you’re going to serve those  
> files if mediatemple won’t install the xsendfile module. If you’re  
> going to stream those using send\_file or send\_data, you’re going to  
> get into some serious problems, no matter how they serve their rails  
> applications. I’d advise you to look into Amazon S3 since you seem to  
> be charging for your videos anyway?
> 
> Best regards
> 
> Peter De Berdt

Hey Peter - this was posted a long time ago…hopefully you are still  
following this thread, but I was wondering what you mean by “you’re  
going to get into some serious problems”? I’m actually using send\_file  
and my Ruby on Rails Container keeps terminating because it has “run out  
of Ram”. This happens when the system is using the send\_file function  
for large files. This may have been what you previously warned me  
about. Do you have any advise or techniques to overcome this issue?

I ended up doing the FTP thing as you advised, but failed to take into  
consideration your send\_file warning. Let me know if you have any  
ideas.

thanks…

---

<div class="post-metadata">

**Author:** ![jimmypalmer](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/j/bbe5ce/32.png) [@jimmypalmer](https://www.ruby-forum.com/u/jimmypalmer)\
**Post date:** [June 10, 2008, 11:23pm UTC](https://www.ruby-forum.com/t/uploading-large-files/133470/6 "2008-06-10T23:23:13Z")

</div>

On 10 Jun 2008, at 01:59, Jimmy P. wrote:

> going to get into some serious problems"? I’m actually using  
> send\_file  
> and my Ruby on Rails Container keeps terminating because it has “run  
> out  
> of Ram”. This happens when the system is using the send\_file function  
> for large files. This may have been what you previously warned me  
> about. Do you have any advise or techniques to overcome this issue?
> 
> I ended up doing the FTP thing as you advised, but failed to take into  
> consideration your send\_file warning. Let me know if you have any  
> ideas.

send\_file and send\_data will load the entire data stream into memory  
before serving it, that’s why your app goes down due to excessive  
memory use.

Now, as I said, there are a couple of options you might consider:

1. Use Apache’s or nginx’ XSendFile module to serve the files. If  
you’re using some kind of shared hosting, where you have no control  
over the actual web, then this method is gonna fall flat on its face,  
because they probably won’t want to install that module for you.  
Apache will then serve a file with a temporary url.

2. Use an external hosting service like Amazon’s S3 to serve the  
files, where you basically offload the file serving to a service  
that’s made just for doing so. Attachment\_fu should be able to help  
you out there.

Best regards

Peter De Berdt

---

<div class="post-metadata">

**Author:** ![jimmypalmer](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/j/bbe5ce/32.png) [@jimmypalmer](https://www.ruby-forum.com/u/jimmypalmer)\
**Post date:** [June 12, 2008, 10:58pm UTC](https://www.ruby-forum.com/t/uploading-large-files/133470/7 "2008-06-12T22:58:55Z")

</div>

thanks paul…

one last question - I’ve got over 18 gigs of video files on my media  
temple server that I will need to store on the amazon s3 storage.  
What’s the best way to transfer these over?

---

<div class="post-metadata">

**Author:** ![jimmypalmer](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/j/bbe5ce/32.png) [@jimmypalmer](https://www.ruby-forum.com/u/jimmypalmer)\
**Post date:** [June 13, 2008, 10:43am UTC](https://www.ruby-forum.com/t/uploading-large-files/133470/8 "2008-06-13T10:43:39Z")

</div>

On 12 Jun 2008, at 22:58, Jimmy P. wrote:

> one last question - I’ve got over 18 gigs of video files on my media  
> temple server that I will need to store on the amazon s3 storage.  
> What’s the best way to transfer these over?

If you have shell access to you media temple account and can run ruby  
on it, using s3sync could be an option:  
[http://s3sync.net/wiki](http://s3sync.net/wiki)

Haven’t used it, but seems to fit your needs.

Best regards

Peter De Berdt