Amazon S3 interface for RoR project

I will soon be using the Amazon S3 interface on a RoR web app that is
about 50% complete at this point. I grabbed the S3 interface from
Amazon and began looking through the code and I found this:

uses Net::HTTP to interface with S3. note that this interface

should only

be used for smaller objects, as it does not stream the data. if you

were

to download a 1gb file, it would require 1gb of memory. also, this

class

creates a new http connection each time. it would be greatly

improved with

some connection pooling.

My jaw kinda dropped open at this point as my web app uploads large
files (up to 200 meg in some cases). Has anyone ran into this issue,
and if so how did you go about getting around this? I’m not sure what
they mean by connection pooling to be honest. I thought that referred
to the database connections.

Thanks

Tevor I. wrote:

My jaw kinda dropped open at this point as my web app uploads large
files (up to 200 meg in some cases).

One correction: this should be “downloads”, not “uploads”.

Trevor,

I would check out s33r… It’s a gem you can install that’s made some
improvements to the Amazon code. The biggest problem I’ve seen so far is
that Net:HTTP has some very small buffer sizes hardcoded, and the s33r
author has written some workarounds for this.

I’m in your situation as well… I’m going to be integrating S3 into a
project soon, and I actually wrote a bunch of extra stuff before I found
s33r. I haven’t used it yet, but I took a look at the code and it seems
that
the author has solved some of the bigger issues.

Let me know how it goes, if you’re able :slight_smile:

Matt

On 11/16/06, Tevor I. [email protected] wrote:


Posted via http://www.ruby-forum.com/.


Thermal Creative
http://blog.thermalcreative.com

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs