# Writing ActiveRecord find result to S3 as a compressed file

**URL:** <https://www.ruby-forum.com/t/writing-activerecord-find-result-to-s3-as-a-compressed-file/191604>\
**Category:** Rails\
**Created:** [July 15, 2010, 8:00am UTC](https://www.ruby-forum.com/t/writing-activerecord-find-result-to-s3-as-a-compressed-file/191604 "2010-07-15T08:00:35Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![interactivenyc](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/85f322/32.png) [@interactivenyc](https://www.ruby-forum.com/u/interactivenyc)\
**Post date:** [July 15, 2010, 8:00am UTC](https://www.ruby-forum.com/t/writing-activerecord-find-result-to-s3-as-a-compressed-file/191604/1 "2010-07-15T08:00:35Z")

</div>

On Heroku, using a delayed\_job, I am trying to periodically write the  
result of a  
ActiveRecord find that has been converted to JSON to an Amazon S3  
bucket as a compressed file.

I have this successfully by writing the JSON results to a temporary  
file using File.open and Zlib:GzipWriter.write and then using  
AWS::S3::S3Object.store to copy the resulting file to S3. The code  
fragment is below.

The problem is the operation aborts I think because it exceeds  
Heroku’s dyno memory or file size contraints when the find returns a  
large number of rows.

Any suggestions how to use streams or some other approach so that  
large find results can be converted to JSON and then stored on S3 as a  
compressed file.

Thanks in advance for any advice.

- Steve W.

--------- code fragment ------  
tmp\_file = “./tmp/#{file\_name}”

# first compress the content in a file

File.open(tmp\_file, ‘w’) do |f|  
gz = Zlib::GzipWriter.new(f)  
gz.write content  
gz.close  
end

```
AWS::S3::Base.establish_connection!(
   :access_key_id => S3_CONFIG['access_key_id'],
   :secret_access_key => S3_CONFIG['secret_access_key'],
   :use_ssl => true

```

)  
AWS::S3::S3Object.store file\_name + “.gz”, open(tmp\_file),  
bucket\_name  
stored = AWS::S3::Service.response.success?  
AWS::S3::Base.disconnect!

File.delete tmp\_file