Faster_csv column type

Ruby
1

Hello all,
I’m using faster_csv to import some csv formatted data to a database.
I’m using FasterCSV.foreach method to add all data but I came across
with a problem I don’t know how to overcome.

First I convert all csv data using the available converters (and also
adding some others) but before I insert the data to the database I need
to know each column type in order to first create the table and then
insert the data.

Does anyone know how to do this?

Thanks in advance.
Best regards,
Migrate

2

On Jul 15, 2007, at 6:44 AM, Hu Ma wrote:

Hello all,

Hello.

Does anyone know how to do this?
Does this give you any fresh ideas?

require “faster_csv”
=> true

data = <<-END
name,age
James,31
Dana,21
END
=> “name,age\nJames,31\nDana,21\n”

FCSV.parse(data, :headers => true, :converters => :numeric) do |row|
?> p row.inject({}) { |h, (k, v)| h.merge(k => v.class) }

end
{“name”=>String, “age”=>Fixnum}
{“name”=>String, “age”=>Fixnum}
=> nil

James Edward G. II

3

Hello James,
Thanks for the answer.

The code that you presented gave me some ideias, however I still don’t
know the best way to overcome some problems:

  • What is the best way to convert a column to boolean values?
  • The csv file can have a column that has both float and integer values
    in it. In this case I want to create a float column in the database.
  • Is there a way to avoid the duplicating of information? The example
    you gave shows the same classes twice.

To better ilustrate the problems here is one sample:
data = <<-END
string,number,boolean
James,32,true
Dana,33.21,false
END

Thanks again.
Best regards,
Migrate

James G. wrote:

On Jul 15, 2007, at 6:44 AM, Hu Ma wrote:

Hello all,

Hello.

Does anyone know how to do this?
Does this give you any fresh ideas?

require “faster_csv”
=> true

data = <<-END
name,age
James,31
Dana,21
END
=> “name,age\nJames,31\nDana,21\n”

FCSV.parse(data, :headers => true, :converters => :numeric) do |row|
?> p row.inject({}) { |h, (k, v)| h.merge(k => v.class) }

end
{“name”=>String, “age”=>Fixnum}
{“name”=>String, “age”=>Fixnum}
=> nil

James Edward G. II

4

On Jul 22, 2007, at 5:11 AM, Hu Ma wrote:

Hello James,

Hello.

Thanks for the answer.

Sure.

The code that you presented gave me some ideias, however I still don’t
know the best way to overcome some problems:

  • What is the best way to convert a column to boolean values?

See below.

  • The csv file can have a column that has both float and integer
    values
    in it. In this case I want to create a float column in the database.

You can force the column to a float. See below.

  • Is there a way to avoid the duplicating of information? The example
    you gave shows the same classes twice.

My example showed the classes for two rows of data. They had the
same classes. Nothing was “duplicated” though.

To better ilustrate the problems here is one sample:
data = <<-END
string,number,boolean
James,32,true
Dana,33.21,false
END

Here’s how I would handle that data:

#!/usr/bin/env ruby -wKU

require “rubygems”
require “faster_csv”

data = <<-END
string,number,boolean
James,32,true
Dana,33.21,false
END

BOOLS = {“true” => true, “false” => false}
parsed = FCSV.parse(
data,
:headers => true,
:header_converters => :symbol,
:converters => [
lambda { |f, i| i.header == :number ? Float(f) : f },
lambda { |f, i| i.header == :boolean ? BOOLS[f] : f }
]
)

p parsed.to_a

>> [[:string, :number, :boolean], [“James”, 32.0, true], [“Dana”,

33.21, false]]

END

Hope that helps.

James Edward G. II