# Import csv with Fields that contain double quote characters

**URL:** <https://www.ruby-forum.com/t/import-csv-with-fields-that-contain-double-quote-characters/199081>\
**Category:** Ruby\
**Created:** [November 14, 2010, 11:14am UTC](https://www.ruby-forum.com/t/import-csv-with-fields-that-contain-double-quote-characters/199081 "2010-11-14T11:14:23Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![okkezSS](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/o/85f322/32.png) [@okkezSS](https://www.ruby-forum.com/u/okkezSS)\
**Post date:** [November 14, 2010, 11:14am UTC](https://www.ruby-forum.com/t/import-csv-with-fields-that-contain-double-quote-characters/199081/1 "2010-11-14T11:14:23Z")

</div>

hi  
i import csv files in ruby.  
it work good, except when a filed has double quote characters- the  
import is failed  
The data fields are in Hebrew  
thanks for help.

---

<div class="post-metadata">

**Author:** ![shir](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/9e8a1a/32.png) [@shir](https://www.ruby-forum.com/u/shir)\
**Post date:** [November 14, 2010, 11:43am UTC](https://www.ruby-forum.com/t/import-csv-with-fields-that-contain-double-quote-characters/199081/2 "2010-11-14T11:43:52Z")

</div>

2010/11/14 הדסה גרינוולד [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid):

> hi  
> i import csv files in ruby.  
> it work good, except when a filed has double quote characters- the  
> import is failed  
> The data fields are in Hebrew  
> thanks for help.

Hi,

What library are you using to import the csv? Are the files in utf-8,  
or are they windows-1255?

Regards,  
Ammar

---

<div class="post-metadata">

**Author:** ![shir](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/9e8a1a/32.png) [@shir](https://www.ruby-forum.com/u/shir)\
**Post date:** [November 14, 2010, 12:05pm UTC](https://www.ruby-forum.com/t/import-csv-with-fields-that-contain-double-quote-characters/199081/3 "2010-11-14T12:05:05Z")

</div>

Ammar A. wrote in post #961324:

> 2010/11/14 הדסה גרינוולד [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid):
> 
> > hi  
> > i import csv files in ruby.  
> > it work good, except when a filed has double quote characters- the  
> > import is failed  
> > The data fields are in Hebrew  
> > thanks for help.
> 
> Hi,
> 
> What library are you using to import the csv? Are the files in utf-8,  
> or are they windows-1255?
> 
> Regards,  
> Ammar

hi  
i do it wuth :fasterCSV,  
and using- windows-1255, and convert it to-utf-8.  
thank you

FasterCSV.open(data\_file,“r”).each\_with\_index do |row,row\_idx|  
logger.info “row\_idx:-----#{row\_idx}------”  
if row\_idx \> 0 #if is not a row of titles  
vals = []  
row.each\_with\_index do |val, col\_idx |  
if col\_idx \> 0 #skip on the first column - a  
number for task (not needed)  
# unless val.blank?  
# val =  
Iconv.iconv(‘utf-8’, ‘Windows-1255’, val)  
# val = val.first  
# end  
logger.info val  
val = Iconv.iconv(  
“utf8”,“Windows-1255”,val)  
logger.info val = val.first  
if col\_idx == indexes[:tsk\_type]  
vals \<\< check\_if\_id(val,task\_type\_ids)  
elsif col\_idx == indexes[:customer]  
vals \<\< check\_if\_id(val,customer\_ids)  
elsif col\_idx == indexes[:area]  
vals \<\< check\_if\_id(val,area\_ids)  
elsif col\_idx == indexes[:status]  
vals \<\< check\_if\_id(val,status\_ids)  
elsif col\_idx == indexes[:date\_for\_perform]  
if val.blank?  
vals \<\< ‘null’  
else  
vals\<\< val  
end  
elsif col\_idx == indexes[:cost]  
if val.blank?  
vals \<\< 0  
else  
vals\<\< val  
end  
else  
val = val.gsub(“'”,“‘’”) unless  
val.blank?  
vals \<\< “‘#{val}’”  
end  
end  
end  
if vals.length \< indexes.length  
(indexes.length - vals.length).times { vals \<\<  
“null” }  
end  
vals \<\<  
“#{book\_id},#{user\_id},‘#{time\_to\_db}’,‘#{time\_to\_db}’”  
values \<\< “(#{vals.join(”,“)})”  
end  
end

```
            unless values.blank?
               begin
                  con = DBConnection.open
                  str_query = "insert into tasks 

```

(date\_for\_perform,customer\_id,order\_number,license\_number,  
company\_name,client\_name,phone\_number,city,street,home\_number,tsk\_type\_id,remarks,area\_id,hour,tsk\_status\_id,  
institute,cost,book\_id,user\_id,created\_at,updated\_at)  
values #{values.join(‘,’)}"  
logger.info str\_query  
con.query(str\_query)  
rescue Exception =\> ex  
return ex  
end  
end  
end

---

<div class="post-metadata">

**Author:** ![shir](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/9e8a1a/32.png) [@shir](https://www.ruby-forum.com/u/shir)\
**Post date:** [November 14, 2010, 1:09pm UTC](https://www.ruby-forum.com/t/import-csv-with-fields-that-contain-double-quote-characters/199081/4 "2010-11-14T13:09:10Z")

</div>

On Sun, Nov 14, 2010 at 1:05 PM, Shir S. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> hi  
> i do it wuth :fasterCSV,  
> and using- windows-1255, and convert it to-utf-8.  
> thank you

Sorry, I couldn’t run that code. Maybe pasting in the forum messed it  
up. I tried a simple test with double quotes and FasterCSV always  
throws:

Illegal quoting on line 2. (FasterCSV::MalformedCSVError)

If that is what you are seeing too, then it’s probably a formating  
issue. The best solution is to export the csv file with legal  
formatting, but if that’s not possible, you can replace the " with  
single quotes as a last resort. But be careful, this is brute force  
and can break other things. For example:

# read the file and replace the " with ’

data = File.read data\_file  
data.gsub!(‘"’, “'”)

# parse instead of open

row\_index = 0  
FasterCSV.parse(data) do |row|  
row\_index +=1  
next if row\_index == 1

converted = row.map {|val| Iconv.iconv(“utf8”,“Windows-1255”,val)}  
puts converted.inspect  
end

If you have a small 2 to 3 line sample of the input (not real data), I  
could try to help more.

Hope that helps,  
Ammar