# Dynamically import csv files

**URL:** <https://www.ruby-forum.com/t/dynamically-import-csv-files/164305>\
**Category:** Rails\
**Created:** [March 31, 2009, 2:27am UTC](https://www.ruby-forum.com/t/dynamically-import-csv-files/164305 "2009-03-31T02:27:38Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![lee](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/l/bc8723/32.png) [@lee](https://www.ruby-forum.com/u/lee)\
**Post date:** [March 31, 2009, 2:27am UTC](https://www.ruby-forum.com/t/dynamically-import-csv-files/164305/1 "2009-03-31T02:27:38Z")

</div>

I’m hoping somebody can give me some direction on how to approach this  
problem. We have various projects with data acquisition in the field.  
These projects produce time series data of variables such as  
temperature, and power usage. I’m using javascript to plot the data.  
Currently I have a form in which the user would submit the project  
name, the type of data (60 minute, 1 minute, hourly, etc.) and the  
names of the variables requested. This generates an ajax request to  
find and parse a csv file to get the variables and send them back to  
the browser to plot.

The csv files are updated daily. New projects are created often. Each  
project has several types of data with many variables and many rows.

I’d like to have the data all in a database, possibly ActiveRecord,  
taking advantage of validations and such. I’ve thought of two  
approaches so far:

- Creating a unique new table for every project and for every type of  
data (e.g. data\_project1\_60minute). This results in creating new  
classes for every table as well, which doesn’t seem optimal. There is  
also the possibility of using the “set\_table\_name” feature of  
activerecord, but I wonder if this is the best way.

- Creating one massive table for every type of data (e.g.  
data\_60minute, with column names project1\_var1, project1\_var2,  
project2\_var1, etc.). Then adding columns whenever a new project is  
added. This also doesn’t seem great since one data’s 60minute has time  
differences between the others. There is also event driven data which  
collects at high frequency for short periods of time, which also  
doesn’t fit this model.

Really it seems like I want some of the features of ActiveRecord, but  
I don’t really want the object relational mapping, and it gets in the  
way by requiring me to make a bunch of new classes.

Thanks for any advice that someone can give.

---

<div class="post-metadata">

**Author:** ![lee](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/l/bc8723/32.png) [@lee](https://www.ruby-forum.com/u/lee)\
**Post date:** [April 4, 2009, 2:07pm UTC](https://www.ruby-forum.com/t/dynamically-import-csv-files/164305/2 "2009-04-04T14:07:05Z")

</div>

Lee,

You could also create two tables, one which describes a particular  
projects column use and another which stores the actual data columns.  
The first metadata table would have one row per project and the data  
in the second would identify which project they are associated with.  
This way you just create a new row for a new project and never change  
the table structures. The validations can be driven by the metadata  
table specifications.

Brendon.