How do I import data FAST? (AR-extensions)

I need to import large amounts of data records into a database with my
rails application. The problem: It’s REALLY SLOW.

I found Zach D.’ ActiveRecord-extensions to be extremely
interesting:
http://www.rubyinside.com/advent2006/17-extendingar.html#import

Quote:“Benchmarks with MySQL’s MyISAM and InnoDb table types without
validations show up to a 48x performance increase.”

BUT: How can I use this with joined tables?

Example:
Supposed “Parent” and “Child” have a has_and_belongs_to_many relation,
i.e. they are linked with a join table “childrens_parents”. Now, I want
to use the import functionality from ar-extension to optimize this:

1000.times {
parent.children.create(:par0 => 42)
}

Problem: Every “create” generates two(!) INSERTs in MySQL:

INSERT INTO children_parents (‘child_id’, ‘parent_id’) VALUES (325648,
480)
INSERT INTO children (‘par0’) VALUES(42)

Is there a way to optimize this with “import” from the AR extensions? Or
is there any other way to do this fast?

Any help is appreciated!

Thanks,
Marc

On Nov 12 2007, 8:55 am, Marc M. [email protected]
wrote:

Problem: Every “create” generates two(!) INSERTs in MySQL:

Thanks,
Marc

Posted viahttp://www.ruby-forum.com/.

Try something like this. The trick is to use arrays to capture the
data for each iteration. Its fast.

require “rubygems”
require “active_record”

#change the following for your database
ActiveRecord::Base.establish_connection(:adapter => “mysql”, :host =>
“localhost”,:database => “railsspace_development”,
:username => “your use name”, :password => “your password”)

class Spec < ActiveRecord::Base;end
class Spectors < Spec

attr_accessor( :user_id, :first_name, :last_name, :gender, :birthdate,
:zip_code)
end
i =0;test_data = YAML.load_file(‘specs2.yml’)
while i < (test_data.length) #number of players

lastname=[];first_name=[];gender=[];birthdate=[];zip_code=[];userid=[]
userid << test_data[i].user_id;first_name <<
test_data[i].first_name
lastname << test_data[i].last_name;gender << test_data[i].gender
birthdate << test_data[i].birthdate;zip_code <<
test_data[i].zip_code
spec = Spec.new;spec.id=i + 1;spec.last_name = lastname.first
spec.first_name = first_name.first.to_s;spec.gender = gender.first
spec.birthdate= birthdate.first;spec.zip_code =
zip_code.first.to_s.to_i
spec.user_id = userid.first.to_s.to_i;spec.save!;i += 1
end

Todd Miller