# Problem with Regular Expression **URL:** \ **Category:** Rails\ **Created:** [December 19, 2008, 8:48pm UTC](https://www.ruby-forum.com/t/problem-with-regular-expression/156054 "2008-12-19T20:48:24Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![rubylicious](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/91b2a8/32.png) [@rubylicious](https://www.ruby-forum.com/u/rubylicious)\ **Post date:** [December 19, 2008, 8:48pm UTC](https://www.ruby-forum.com/t/problem-with-regular-expression/156054/1 "2008-12-19T20:48:24Z")
Why would the following regular expression produce the results below? \*\*\*\*input: john,doe,“1,200.99”,training,12345 \*\*_code executed on input: values = line.split(/("\d,\d+.\d_")|,/) \*\*\*\*output: [“john”,“doe”,"",“1,200.99”,"",“training”,“12345”] What’s up with the two empty array indexes that are generated during the split surrounding my dollar value? The regular expression is pretty explicit and the input string doesn’t contain anything that should cause them when run through the expression, yet there they are so obviously I’m doing something wrong in my regexp. Help. ---
**Author:** ![rubylicious](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/91b2a8/32.png) [@rubylicious](https://www.ruby-forum.com/u/rubylicious)\ **Post date:** [December 19, 2008, 8:59pm UTC](https://www.ruby-forum.com/t/problem-with-regular-expression/156054/2 "2008-12-19T20:59:11Z")
On 19 Dec 2008, at 19:48, Corey M. \ wrote: > [“john”,“doe”,“”,“1,200.99”,“”,“training”,“12345”] Pretend you’re split. You see the comma after doe so you split. Then you see the other part of your regex and so you split again, resulting in the empty array Fred ---
**Author:** ![rubylicious](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/91b2a8/32.png) [@rubylicious](https://www.ruby-forum.com/u/rubylicious)\ **Post date:** [December 19, 2008, 9:08pm UTC](https://www.ruby-forum.com/t/problem-with-regular-expression/156054/3 "2008-12-19T21:08:21Z")
Frederick C. wrote: > Pretend you’re split. You see the comma after doe so you split. Then > you see the other part of your regex and so you split again, resulting > in the empty array > > Fred Ok, but the pipe character stands for alternation much like an OR so once it sees doe and splits, the next part of the csv input it evaluates should match to the monetary portion of the regex so it splits around it. Maybe I’m just not getting somethings because I want it to split my (comma delimited) input line based on comma or when a value is wrapped with double quotes. I can always handle the extra empty array positions by simple performing a array.delete\_if with a block that looks for empty strings but that seems like overkill if I could handle it with the regex instead. Sorry if I’m still not getting the point. ---
**Author:** ![rubylicious](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/91b2a8/32.png) [@rubylicious](https://www.ruby-forum.com/u/rubylicious)\ **Post date:** [December 19, 2008, 9:19pm UTC](https://www.ruby-forum.com/t/problem-with-regular-expression/156054/4 "2008-12-19T21:19:07Z")
On Dec 19, 2008, at 2:58 PM, Frederick C. wrote: > > split surrounding my dollar value? > > > > The regular expression is pretty explicit and the input string > > doesn’t > > contain anything that should cause them when run through the > > expression, > > yet there they are so obviously I’m doing something wrong in my > > regexp. > > Help. I think you actually get: =\> [“john”, “doe”, “”, “"1,200.99"”, “”, “training”, “12345”] as your output. (At least I do.) However, it looks like your input is CSV so why not let a CSV library do the heavy-lifting for you: irb\> require ‘rubygems’ =\> true irb\> require ‘fastercsv’ =\> true irb\> FasterCSV.parse\_line(line) =\> [“john”, “doe”, “1,200.99”, “training”, “12345”] Note: FasterCSV becomes the CSV in the standard library for Ruby 1.9, but it’s a gem before that so look at the rdocs/ri for whatever you’re using. -Rob Rob B. [http://agileconsultingllc.com](http://agileconsultingllc.com) removed\_email\_address@domain.invalid ---
**Author:** ![rubylicious](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/91b2a8/32.png) [@rubylicious](https://www.ruby-forum.com/u/rubylicious)\ **Post date:** [December 19, 2008, 9:26pm UTC](https://www.ruby-forum.com/t/problem-with-regular-expression/156054/5 "2008-12-19T21:26:28Z")
Rob B. wrote: > However, it looks like your input is CSV so why not let a CSV library > do the heavy-lifting for you: Primarily because this is a budget application I’m writing and the data coming from Accounts Payable (.csv files) need to be read in and then massaged some before moving it into the database. A lot of the massaging relates to categorizing transactions and associating other information with a record that isn’t a part of the original input file. So, I’m manually parsing this data into an array for active record column attribute assignment and the like. Unless fasterCSV reads a file in for me and give me the data in an array I can use in my model or controller for further processing, it doesn’t fit my needs. So forgive my ignorance if it does do this and I’m simply not aware. However, if it does, then woohoo and I’ll be pulling down the gem pronto. ---
**Author:** ![rubylicious](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/91b2a8/32.png) [@rubylicious](https://www.ruby-forum.com/u/rubylicious)\ **Post date:** [December 19, 2008, 10:07pm UTC](https://www.ruby-forum.com/t/problem-with-regular-expression/156054/6 "2008-12-19T22:07:12Z")
On Dec 19, 2008, at 3:26 PM, Corey M. wrote: > file. > So, I’m manually parsing this data into an array for active record > column attribute assignment and the like. > > Unless fasterCSV reads a file in for me and give me the data in an > array > I can use in my model or controller for further processing, it doesn’t > fit my needs. So forgive my ignorance if it does do this and I’m > simply > not aware. However, if it does, then woohoo and I’ll be pulling down > the gem pronto. Then start now! gem install fastercsv I was just using one of the lower-level methods to parse your input line. I’ll let you read the docs[1] for yourself, but it will read the file and give the data back to you as an array or a hash (you choose). You can configure whether there are first-row headers, too. -Rob [1] [http://fastercsv.rubyforge.org/](http://fastercsv.rubyforge.org/) Rob B. [http://agileconsultingllc.com](http://agileconsultingllc.com) removed\_email\_address@domain.invalid ---
**Author:** ![rubylicious](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/91b2a8/32.png) [@rubylicious](https://www.ruby-forum.com/u/rubylicious)\ **Post date:** [December 20, 2008, 8:04am UTC](https://www.ruby-forum.com/t/problem-with-regular-expression/156054/7 "2008-12-20T08:04:35Z")
On Dec 19, 3:08 pm, Corey M. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > it. Maybe I’m just not getting somethings > Maybe not. 🙂 Frederick is absolutely right. Here’s what I believe > happens when split is working on your input string. The split routine starts parsing at the beginning of the string (I will represent the cursor in these examples with a #.) #john,doe,“1,200.99”,training,12345 =\> [] At the moment, no results have been yielded, so the resulting array (after the =\>) is still empty. Split runs until it sees a delimiter. john#,doe,“1,200.99”,training,12345 =\> [‘john’] Split discards the delimiter. john,#doe,“1,200.99”,training,12345 =\> [‘john’] Repeat for the next element (‘doe’) and delimiter (‘,’). john,doe,#“1,200.99”,training,12345 =\> [‘john’, ‘doe’] Having just read a delimiter (‘,’), split is now prepared to read the next value. But it doesn’t see a value – it sees another delimiter (‘“1,200.99”’, which matches your regex and is therefore considered a delimiter just like ‘,’). Split therefore writes an empty array element. john,doe,#“1,200.99”,training,12345 =\> [‘john’, ‘doe’, ‘’] Split skips over the delimiter – but since the regexp contains _capturing parentheses_, split inserts the parenthesized portion of the match (which happens to match the whole delimiter) into the result set. john,doe,“1,200.99”#,training,12345 =\> [‘john’, ‘doe’, ‘’, ‘“1,200.99”’] Split sees another delimiter (‘,’) with no intervening value, so it writes another empty string. This time, the regexp that matches the delimiter has no parentheses, so split does not put it into the result set. john,doe,“1,200.99”,#training,12345 =\> [‘john’, ‘doe’, ‘’, ‘“1,200.99”’, ‘’] Split reads the last two values, skipping the delimiter in between. john,doe,“1,200.99”,training,12345# =\> [‘john’, ‘doe’, ‘’, ‘“1,200.99”’, ‘’, ‘training’, ‘12345’] Does that make it clearer? Since you defined the “1,200.99” string as a delimiter, the fact that it appears in the result set at all is actually accidental, and due 100% to the parentheses that you put around it in the regexp. Remember, the regexp in split defines the pattern for the _delimiter_, not the _values_. If you’re still confused about what the parentheses do, here’s a minimal example: ‘aabbcc’.split(/b+/) =\> [‘aa’, ‘cc’] ‘aabbcc’.split(/(b+)/) =\> [‘aa’, ‘bb’, ‘cc’] > because I want it to split my > (comma delimited) input line based on comma or when a value is wrapped > with double quotes. I hate to break it to you, but a single regexp alone cannot handle this kind of parsing, because the meaning of a comma is _state- dependent_ – that is, a comma either acts as a delimiter or not, depending on whether an odd or even number of double quotes have been encountered since the beginning of the string. And regexps are not state-dependent, so that kind of logic is outside the capabilities of a regexp. You’ll need to write some higher-level routines in Ruby – or, as others have suggested, simply use an off-the-shelf CSV parser. (I’d choose the latter approach if I were you – parsers are a pain to get right!) ## Best, Marnen Laibow-Koser removed\_email\_address@domain.invalid [http://www.marnen.org](http://www.marnen.org)