I’m about to start on my first Rails project, and I could use some
advice. I’m going to build a “Data Blender” to replace a lot of
frantic work that gets done here by hand using Microsoft Access.
Using the Data Blender, you’ll be able to import our variety of 30,000
to 200,000 record data files into mySQL, then slice off parts, two
counts, export, etc… Initially it will make it easy to save and
modify import and export formats (a lot of the data I get have
relatively similar but not the same formats). Eventually, you’ll be
able to build a simple data recipes – a very stripped down version of
Yahoo pipes. For storing items such as import/export formats, I’ll
use Active Record. But for data files I’m importing on-the-fly, is it
worth it, or does it make more sense to do it all in SQL? Any advice
would be greatly appreciated.
Thanks,
Anders