Best approach for importing data from a file

Rails
1

Hi All,

I’ve got a basic Rails app started. One wrinkle is that the user
wants ideally to be able to browse the files system (a la Windows
Explorer) to select a file and initiate the importation of data from
the file.

All files selected will be of a predetermined fixed format, e.g. CSV
with Name, Address, etc.

It seems to me that the browser is isolated from the file system,
albeit Rails’ internals know how to get to it. Is there a way to
implement the above-described functionality in a Rails app? Can YAML
be used for such functionality?

The only approach I see right now is a Ruby database-updater using
perhaps Active Record with ar-extensions or Active Scaffold.

Any guidance would be most appreciated.

Best wishes,
Richard

2

A Google search for

rails import csv

returns several promising results.

Regards,
Craig

3

Hi All,

I’ve thought more about this and think I know how to proceed with a
Rails-integrated approach.

Thanks anyway if you’ve looked at the original post.

Best wishes,
Richard

On Oct 23, 10:23 pm, RichardOnRails

4

Thanks, Craig. That search suggestion does help.

Best wishes,
Richard

On Oct 24, 12:15 am, “Craig D.” [email protected]

5

Take a look at attachment_fu.

On Oct 23, 4:23 pm, RichardOnRails

6

Hi Rick,

I’ve been busy, so I haven’t checked for responses to my posts for a
week or so. Sorry for the delay in this response.

Thanks for your response. I just ordered Advanced Rails Recipes and
looked at the attachment_fu.rb from the book. I’m too weak in Rails
to see how that’s useful for my project.

I think I’ve concocted a solution to my main concern: how to access a
CVS file from the file-system from within a Rails app. I now believe
I can:

  1. get the filenames housed in a specific directory from within a
    controller using normal Ruby programming.
  2. display those filenames as a list in a view and allow the user to
    select any one of then
  3. open the selected file and read its records, one by one
  4. extract the fields from each cvs-formatted record and update the
    database with the contents of those fields.

Does that sound sensible to you?

Best wishes,
Richard

7

Hey Richard,

Sorry it’s been so long, I seem to have dropped this thread.

  1. My thought about attachment_fu is that it allows you to pull files
    that are local to your “rails client” machine and bring them into the
    rails application space. One way it’s used is to enable rails users
    to post their own images onto a site for sharing with the community.
    There is a browser included that works much like the usual file
    browser to aid with locating / selecting file for inclusion.

I’m not sure where your csv files live right now but if they’re not
somewhere below RAILS_ROOT + “public” then you’ll need to figure out
how to get them there. Attachment_fu can help here if the files are
user generated - field data and the like - that the user will upload
into the system.

  1. The next thing is moving the data into your Model(s). I’m not sure
    how comfortable you are with the quality of the data but I guarantee
    that, no matter what approach you take, you’ll need to constantly work
    on data validation.

You’ll probably do best if you start with one model for each file
type. That way you can use the model.rb file to validate, validate,
validate,… the data on it’s way to the database.

  1. Of course if it’s your own data and you know it’s always complete
    and correct, you can take the migration file approach. It’s a little
    more raw than what is generally considered socially acceptable in the
    RoR community. The real drawback is then specific database methods
    leak in past the abstraction boundary. Notice the code for postgresql
    differs from that for mysql - and we haven’t even looked at sqlite3 or
    db2 or …:stuck_out_tongue:

class CreateStateAbbrevs < ActiveRecord::Migration
def self.up
create_table :state_abbrevs do |t|
t.string :name
t.string :abbrev
end
csv_file = “#{RAILS_ROOT}/db/migrate/abbr_state.csv”
fields = ‘(name, abbrev)’

# PostgreSQL load from file...
execute "COPY state_abbrevs #{fields} FROM '#{csv_file}' WITH CSV"

# MySQL load from file...
# execute "LOAD DATA INFILE '#{csv_file}' INTO TABLE state_abbrevs

FIELDS " +
# “TERMINATED BY ‘,’ OPTIONALLY ENCLOSED BY “””" " +
# "LINES TERMINATED BY ‘\n’ " + fields

end

def self.down
drop_table :state_abbrevs
end
end

So many buttons and so little time,
Rick

On Nov 13, 5:11 am, RichardOnRails