# Advice on handling malformed CSV with FasterCSV

**URL:** <https://www.ruby-forum.com/t/advice-on-handling-malformed-csv-with-fastercsv/79768>\
**Category:** Ruby\
**Created:** [December 18, 2006, 9:43pm UTC](https://www.ruby-forum.com/t/advice-on-handling-malformed-csv-with-fastercsv/79768 "2006-12-18T21:43:21Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![weyus](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/w/7ab992/32.png) [@weyus](https://www.ruby-forum.com/u/weyus)\
**Post date:** [December 18, 2006, 9:43pm UTC](https://www.ruby-forum.com/t/advice-on-handling-malformed-csv-with-fastercsv/79768/1 "2006-12-18T21:43:21Z")

</div>

All,

FasterCSV 1.0.0  
Ruby 1.8.4

I am writing functionality to replace an an older system. Part of this  
system accepts uploaded CSV files, parses, and displays their contents.  
When confronted with a file including one line that looks like this:

“data”,'otherdata"

the legacy app. happily parses this into two fields with values data and  
otherdata.

However, FasterCSV fails because of the quote mismatch on the 2nd field  
and throws a MalformedCSVError when you attempt to manipulate the object  
returned from FasterCSV.open().

I’ve taken a look at the code and I can see where the quote matching is  
enforced and so I have a pretty good idea of how to “fix” this (which I  
am only even considering because I’m trying to match the legacy  
functionality).

I need some advice. My questions are:

1. Is there a way to get FasterCSV to accept input like the above? How  
can I set up a custom parsing scheme?
2. Should I modify the FasterCSV parser directly to accept this sort of  
input?
3. Should I pre-process the file to eliminate/modify these sorts of  
cases to improve the likelihood of a successful FasterCSV parse?

Thanks,  
Wes

---

<div class="post-metadata">

**Author:** ![weyus](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/w/7ab992/32.png) [@weyus](https://www.ruby-forum.com/u/weyus)\
**Post date:** [December 18, 2006, 10:36pm UTC](https://www.ruby-forum.com/t/advice-on-handling-malformed-csv-with-fastercsv/79768/2 "2006-12-18T22:36:36Z")

</div>

Wes G. wrote:

/ …

> I need some advice. My questions are:
> 
> 1. Is there a way to get FasterCSV to accept input like the above? How  
> can I set up a custom parsing scheme?
> 2. Should I modify the FasterCSV parser directly to accept this sort of  
> input?
> 3. Should I pre-process the file to eliminate/modify these sorts of  
> cases to improve the likelihood of a successful FasterCSV parse?

I strongly recommend that you normalize the database to conform to CSV  
conventions, before performing any processing. This is much to be  
preferred  
to rewriting a standard library in such a way that it accepts the  
malformed  
data.

The original database doesn’t meet normal CSV conventions. If it is  
expected  
to exist for any length of time in its present form, you may find  
yourself  
rewriting a lot of parsers to get them to accept it, or you can fix the  
database itself, once. I think the latter approach makes more sense.

---

<div class="post-metadata">

**Author:** ![weyus](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/w/7ab992/32.png) [@weyus](https://www.ruby-forum.com/u/weyus)\
**Post date:** [December 18, 2006, 10:43pm UTC](https://www.ruby-forum.com/t/advice-on-handling-malformed-csv-with-fastercsv/79768/3 "2006-12-18T22:43:31Z")

</div>

Paul L. wrote:

> I strongly recommend that you normalize the database to conform to CSV  
> conventions, before performing any processing. This is much to be  
> preferred  
> to rewriting a standard library in such a way that it accepts the  
> malformed  
> data.

I agree.

Thanks, Paul.

Wes