Forum: Ruby on Rails csv import - encoding problem

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
78d56bb0f91c899d40f1767b3040d825?d=identicon&s=25 bingo bob (bingobob)
on 2009-02-03 12:49
I'm trying to load a CSV file into my rails app using a migration and
FasterCSV and running into problems.

I've had this working before when the CSV file has been saved out of
excel, I'd like to do it with numbers now but I can't quite get it to
work as is. I can't believe that excel does a better job with CSV than
numbers (I don't want to believe it)! The option I'm using in numbers to
do the export, is Share -> Export as CSV (set the encoding to UTF8, I
could try with a different option in numbers if that might help).

This seems like an encoding problem, is there a way I can handle this in
the migration.

Here's the error message the migration spits out….

Unquoted fields do not allow \r or \n (line 2).


-----------

Be grateful for any help at all.

-----

And here's the migration file….



require 'fastercsv'
class LoadData < ActiveRecord::Migration

  def self.up
    count=0
    FasterCSV.foreach("#{RAILS_ROOT}/db/migrate/fixtures/fire.csv",
:row_sep => "\r") do |row|
      forename,jobtitle,initials,surname,departmentname,jobfunction,companyname,addressline1,addressline2,addressline3,addressline4,town,postalcode,country,region,telephone,fax,email,web,persontelephone,personfax,personemail
= row
      # name,url = row
      # Contact.create(:name => name, :url => url)
      Contact.create(

      :forename => forename,
      :job_title => jobtitle,
      :initials => initials,
      :surname => surname,
      :department => departmentname,
      :job_role => jobfunction,
      :company_name => companyname,
      :address1 => addressline1,
      :address2 =>  addressline2,
      :address3 => addressline3,
      :address4 => addressline4,
      :town => town,
      :postcode => postalcode,
      :country => country,
      :region => region,
      :telephone => telephone,
      :fax => fax,
      :email => email,
      :web => web,
      :personal_phone => persontelephone,
      :personal_fax => personfax,
      :personal_email => personemail

      )

      count += 1
      # puts
      # puts count.to_s + " Name: " + row[1]
      # puts row.to_yaml
      # puts
      # puts row.inspect
      # puts row[1]
      # puts record.url
    end
    puts count.to_s + " records inserted."
    records = Contact.find(:all)
    # puts records.count
  end

  def self.down
    puts "not doing anything"
  end

end
78d56bb0f91c899d40f1767b3040d825?d=identicon&s=25 bingo bob (bingobob)
on 2009-02-03 16:08
anyone help with this.

I'm not sure what's happening here. Maybe my data contains line feeds
within the cells of data itself and that's throwing it out?

Another thought is that I could go into numbers and do a find replace on
the carriage return character...just not sure?

What does this mean?

Unquoted fields do not allow \r or \n (line 2).
1128ccdc6b414a39e38f00acca680b8d?d=identicon&s=25 unknown (Guest)
on 2009-02-03 19:26
(Received via mailing list)
put double quotes around any fields that contain a return character or a
comma. Not sure if you have to escape return characters or not, but
you'll
find out :-)

Be sure to escape any double quotes

On Feb 3, 2009 7:08am, bingo bob <rails-mailing-list@andreas-s.net>
wrote:
> within the cells of data itself and that's throwing it out?
>
> --~--~---------~--~----~------------~-------~--~----~
>
> You received this message because you are subscribed to the Google
Groups "Ruby on Rails: Talk" group.
>
> To post to this group, send email to rubyonrails-talk@googlegroups.com
>
> To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe@googlegroups.com
>
> For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
78d56bb0f91c899d40f1767b3040d825?d=identicon&s=25 bingo bob (bingobob)
on 2009-02-03 19:30
This command did it for me, in the terminal...

tr -d '\n' < edit_unicode_utf8.csv > fire_new.csv

stripped out \n i suppose.
1128ccdc6b414a39e38f00acca680b8d?d=identicon&s=25 unknown (Guest)
on 2009-02-03 19:36
(Received via mailing list)
Looks like Ruby has some support for writing csv files!

http://www.ruby-doc.org/stdlib/libdoc/csv/rdoc/cla...


On Feb 3, 2009 10:30am, bingo bob <rails-mailing-list@andreas-s.net>
wrote:
>
>
> You received this message because you are subscribed to the Google
Groups "Ruby on Rails: Talk" group.
>
> To post to this group, send email to rubyonrails-talk@googlegroups.com
>
> To unsubscribe from this group, send email to
rubyonrails-talk+unsubscribe@googlegroups.com
>
> For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
This topic is locked and can not be replied to.