Forum: Ruby on Rails Migration: upload a large set of data via a separate sql fil

Announcement (2017-05-07): is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see and for other Rails- und Ruby-related community platforms.
A0bae8a464f7bdae361c4a07e4f84cd8?d=identicon&s=25 Chuong Huynh (Guest)
on 2007-05-12 01:05
(Received via mailing list)
Hi all,

I need to upload a large set of data (the cvs file is about 70MB) into
some look up tables. Using standard migration api would be tedious. I
know that there's a way to export data into YAML file and populate
data using this file. However, I think this is still heavy.

Is there a way to upload data using a separate sql file in migration?

971708ecf1c0b3cada6cf98ea58611fd?d=identicon&s=25 Val (Guest)
on 2007-05-12 01:18
(Received via mailing list)
You can use the approach described here for loading a straight SQL
file from migrations:

Ef3aa7f7e577ea8cd620462724ddf73b?d=identicon&s=25 Rob Biedenharn (Guest)
on 2007-05-12 20:13
(Received via mailing list)
On May 11, 2007, at 7:04 PM, Chuong Huynh wrote:

> Thanks
This is directly from one of my migrations:

   # Get all that research data loaded!
   def self.up
     say_with_time("Create Tables and Indexes...")
{ create_tables_and_indexes }
     say "Reset all models";                         reset_all_models
     say_with_time("Import all data...")
{ import_data               }

The  import_data  method used FasterCSV to load a 1.6Mb CSV file into
three tables with ActiveRecord models.  The thing to remember is that
you can do *anything* in your migrations.  If the data were already
in an SQL file, use the Revolution Health approach.  If your data is
in CSV, use FasterCSV.  The "migration api" only deals directly with
the schema changes.

Do what makes sense in your situation.


Rob Biedenharn
This topic is locked and can not be replied to.