A ruby script to convert all csv files in a folder to json

Ruby
1

I am trying to build a script to convert all .csv files in a folder to
.json. I have a script with me where I use FasterCSV to convert
individual CSVs to JSON i.e like this:

Code:
#!/usr/local/bin/ruby
require “rubygems”
require “fastercsv”
require “json”

jsonFile = File.open(“some.json”,‘w’);
jsonData = {}

FasterCSV.foreach(“some.csv”,
:headers => true, :header_converters => :symbol) do |row|

puts “Row is #{row}”

jsonData[somename] = {
:name => row[2],
:length => row[3],
:lat => row[4],
:long => row[5],
};

puts “data is #{jsonData[jsonData.length-1]}”

jsonFile.write(JSON.pretty_generate(jsonData));

But how can I change this to search all csv files in a folder and
generate corresponding JSON.

  1. One other thing I am stuck with is how to generate nested JSON i.e

currently my JSON is like this:

Code:
{
“NAME1”: {
“LAT”: “37.847048”,
“Name”: “SDFSDG”,
“length”: “0.03”,
“LON”: “-123.3433”
}
}{
“NAME1”: {
“LAT”: “37.847048”,
“Name”: “SDFSDFSDAFDG”,
“length”: “0.03”,
“LON”: “-123.32334433”
}
}

One issue with above is that NAME1 is redundant. So I want to add it to
the top and then have 2 seperate fields with other data like “lat”,
“lon”, “length”.

I would really appreciate the help.

2

Dear Varun J.,

first how to get a list of files in ruby?

  • solution 1: use ARGV and use shell globbing, eg ruby foo.rb *.csv
  • solution 2: use Dir["*.csv"]

Thus

def convert(in, out)

in ruby 1.9 or so FasterCSV should be the same as ‘csv’ AFAIK.

end

Dir[’*.csv’].each do |in_file|
convert(from, in_file.gsub(’.csv’,’.json’))
end

How to create nested JSON? Learn about how to nest hashes. Eg:

hash = Hash.new
hash[“key”] = Hash.new
hash[“key”][“sub-key”] = ‘value’

The turn hash into json - there are many different libraries.
Eg require ‘json’; hash.to_json

thus:

File.open(out, “wb”) { |file| file.write(hash.to_json) }

How to turn your csv into nested hashes? You didn’t provide enough
information - no way to answer that.

Marc W.

3

Marc W. wrote in post #1121600:

Dear Varun J.,
How to create nested JSON? Learn about how to nest hashes. Eg:

hash = Hash.new
hash[“key”] = Hash.new
hash[“key”][“sub-key”] = ‘value’

The turn hash into json - there are many different libraries.
Eg require ‘json’; hash.to_json

Thanks Marc. I will take a look at how to nest hashes.

4

On Sep 16, 2013, at 8:14 PM, Varun J. [email protected] wrote:

I am trying to build a script to convert all .csv files in a folder to
.json.

There are several ways to do this. You could simply use a bash script:

#!/bin/bash
for csv in *.csv; do
ruby csv_to_json $csv
end

in which case, you’d need to modify your script to pick up the file from
ARGV:

csvfilename = ARGV.shift
jsonfilename = csvfilename.sub(/.csv$/i, ‘.json’)

jsonFile=File.open(jsonfilename.‘w’)

FasterCSV.foreach(csvfilename,

You could also do that for bash stuff directly in the script with the
command line:

ruby csv_to_json *.csv

in which case, your script will become:

while (csvfilename = ARGV.shift) do
jsonfilename = csvfilename.sub(/.csv$/i,‘.json’)

. # and so on, inside the while loop

end

Or pull everything inside:

csvfiles = Dir[‘*.csv’]
while (csvfilename = csvfiles.shift) do
jsonfilename # like above

end

jsonData = {}
};
currently my JSON is like this:
“NAME1”: {
“lon”, “length”.
Why not just make them elements in a JSON array:

{[{
“LAT”: “37.847048”,
“Name”: “SDFSDG”,
“length”: “0.03”,
“LON”: “-123.3433”
},
{
“LAT”: “37.847048”,
“Name”: “SDFSDFSDAFDG”,
“length”: “0.03”,
“LON”: “-123.32334433”
},

]}

code for that:

jsonData = [] # Array instead of hash

FasterCSV.foreach(“some.csv”, :headers => true, :header_converters =>
:symbol) do |row|
jsonData<< {
:name => row[2],
:length => row[3],
:lat => row[4],
:long => row[5],
};
end

Alternatively, you can keep the hash of hashes and do this:

jsonData = {}
FasterCSV.foreach(“some.csv”, :headers => true, :header_converters =>
:symbol) do |row|
jsonData[row[2]]= {
:name => row[2],
:length => row[3],
:lat => row[4],
:long => row[5],
};
end

Assuming the “name” column in some.csv is uniq.