Massive fixtures will kill

Hey there,

I’ve just tried to load a massive CSV fixture file of about 2700000
lines (cites, locations etc). Not surprisingly it died half way through.

Does anyone know how to load such an amount of data? Maybe splitting it
into multiple files?

Sorry for such a newb question… but, well, I am (to rails).

John

El 8 Apr 2011, a les 22:45, John W. [email protected] va
escriure:

Hey there,

I’ve just tried to load a massive CSV fixture file of about 2700000
lines (cites, locations etc). Not surprisingly it died half way through.

Does anyone know how to load such an amount of data? Maybe splitting it
into multiple files?

Sorry for such a newb question… but, well, I am (to rails).

Do you need those many records for your test suite?

John W. wrote in post #991838:

I’ve just tried to load a massive CSV fixture file of about 2700000
lines (cites, locations etc). Not surprisingly it died half way through.

Does anyone know how to load such an amount of data? Maybe splitting it
into multiple files?

Sorry for such a newb question… but, well, I am (to rails).

Don use fixtures, not even when just starting out. Instead choose one of
the excellent test data factory gems to generate test data. And, only
generate test data specifically for the tests you write. You’re not
going to write 2700000 test on city data. Just a few examples are all
you’ll need for your test suite. Generate those as you need them with
your factory.

https://github.com/thoughtbot/factory_girl
https://github.com/notahat/machinist

Does anyone know how to load such an amount of data? Maybe splitting it
into multiple files?

If you are loading it into a MySQL database, use the LOAD DATA INFILE
command. It’s very fast and pretty flexible in terms of allowing you to
define the record separators, file separators, and field mappings.

  • D

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs