# Regex for splitting string

**URL:** <https://www.ruby-forum.com/t/regex-for-splitting-string/93648>\
**Category:** Rails\
**Created:** [April 5, 2006, 2:19pm UTC](https://www.ruby-forum.com/t/regex-for-splitting-string/93648 "2006-04-05T14:19:54Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![yash](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/y/f17d59/32.png) [@yash](https://www.ruby-forum.com/u/yash)\
**Post date:** [April 5, 2006, 2:19pm UTC](https://www.ruby-forum.com/t/regex-for-splitting-string/93648/1 "2006-04-05T14:19:54Z")

</div>

Hi

We have a search website where the user can type in individual words  
separated by spaces and/or phrases enclosed in single or double quotes.  
We are looking for a way to obtain a list of words and phrases from the  
search string.  
Can someone help?

Thanks,  
Yash

---

<div class="post-metadata">

**Author:** ![yash](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/y/f17d59/32.png) [@yash](https://www.ruby-forum.com/u/yash)\
**Post date:** [April 5, 2006, 3:20pm UTC](https://www.ruby-forum.com/t/regex-for-splitting-string/93648/2 "2006-04-05T15:20:43Z")

</div>

Yash wrote:

> Hi
> 
> We have a search website where the user can type in individual words  
> separated by spaces and/or phrases enclosed in single or double quotes.  
> We are looking for a way to obtain a list of words and phrases from the  
> search string.  
> Can someone help?

string.scan(/\w+/)  
to give an array of words, or  
string.split(/\W+/)  
to split on non-words. You can string.gsub(/["’]/, ‘’) if you want to  
get rid of quotes.

---

<div class="post-metadata">

**Author:** ![yash](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/y/f17d59/32.png) [@yash](https://www.ruby-forum.com/u/yash)\
**Post date:** [April 5, 2006, 4:46pm UTC](https://www.ruby-forum.com/t/regex-for-splitting-string/93648/3 "2006-04-05T16:46:58Z")

</div>

If the input string is:  
Java Ruby ‘Ruby on rails’ “software development” “technology”

The list of words should be:  
Java  
Ruby  
Ruby on rails  
software development  
technology

With your approach, the result will be:  
Java  
Ruby  
Ruby  
on  
rails  
software  
development  
technology

Alex Y. wrote:

> Yash wrote:
> 
> > Hi
> > 
> > We have a search website where the user can type in individual words  
> > separated by spaces and/or phrases enclosed in single or double quotes.  
> > We are looking for a way to obtain a list of words and phrases from the  
> > search string.  
> > Can someone help?
> 
> string.scan(/\w+/)  
> to give an array of words, or  
> string.split(/\W+/)  
> to split on non-words. You can string.gsub(/["’]/, ‘’) if you want to  
> get rid of quotes.

---

<div class="post-metadata">

**Author:** ![yash](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/y/f17d59/32.png) [@yash](https://www.ruby-forum.com/u/yash)\
**Post date:** [April 5, 2006, 5:41pm UTC](https://www.ruby-forum.com/t/regex-for-splitting-string/93648/4 "2006-04-05T17:41:52Z")

</div>

I’m not enough of a regex guru to do it in one, so I’d probably do it in  
two:

/"([^"]+)"|(’[^’]+)’/ to grab the quotes … replace occurences with ‘’  
in  
original string …

Then use the \w|\W from below to get individual tokens … at sompoint  
cleaning the original string of anything you constitute garbage.

---

<div class="post-metadata">

**Author:** ![yash](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/y/f17d59/32.png) [@yash](https://www.ruby-forum.com/u/yash)\
**Post date:** [April 5, 2006, 5:41pm UTC](https://www.ruby-forum.com/t/regex-for-splitting-string/93648/5 "2006-04-05T17:41:51Z")

</div>

Yash wrote:

> > > We have a search website where the user can type in individual  
> > > words
> 
> > > separated by spaces and/or phrases enclosed in single or double  
> > > quotes.
> 
> > > We are looking for a way to obtain a list of words and phrases from  
> > > the
> 
> > > search string.

> If the input string is:  
> Java Ruby ‘Ruby on rails’ “software development” “technology”
> 
> The list of words should be:  
> Java  
> Ruby  
> Ruby on rails  
> software development  
> technology

> > example = ‘some text and ‘some inside’ test “double quotes”’

Using the CSV module:

> > require ‘csv’  
> > CSV::parse\_line(example, ’ ‘)  
> > =\> [“some”, “text”, “and”, “'some”, "inside’", “test”, “double quotes”]

Fairly elegant, but doesn’t handle single quotes like you want

or

> > example.split( / \*"’["’] \*| / )  
> > =\> [“some”, “text”, “and”, “some inside”, “test”, “double quotes”]

Which seems to be more like you want.

Hope that helps.

---

<div class="post-metadata">

**Author:** ![yash](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/y/f17d59/32.png) [@yash](https://www.ruby-forum.com/u/yash)\
**Post date:** [April 5, 2006, 5:44pm UTC](https://www.ruby-forum.com/t/regex-for-splitting-string/93648/6 "2006-04-05T17:44:55Z")

</div>

Oops, I seem to be capturing the opening single quote … should move  
that  
…

/"([^"]+)"|’([^’]+)’/

---

<div class="post-metadata">

**Author:** ![yash](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/y/f17d59/32.png) [@yash](https://www.ruby-forum.com/u/yash)\
**Post date:** [April 5, 2006, 6:24pm UTC](https://www.ruby-forum.com/t/regex-for-splitting-string/93648/7 "2006-04-05T18:24:52Z")

</div>

On 4/5/06, James L. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> If you don’t mind having some array elements that are all whitespace  
> you can drop the “find\_all” part.

Of course, the second I hit “send” I realized that I pasted the wrong  
regex.

a = s.split(/(“._?")|('._?')|((?=[^”‘])\w+(?=[^"’]))/).find\_all {|x|  
x.match(/\w+/)}  
=\> [“Java”, “Ruby”, “‘Ruby on rails’”, “"software development"”,  
“"technology  
"”]

– James

---

<div class="post-metadata">

**Author:** ![yash](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/y/f17d59/32.png) [@yash](https://www.ruby-forum.com/u/yash)\
**Post date:** [April 5, 2006, 6:21pm UTC](https://www.ruby-forum.com/t/regex-for-splitting-string/93648/8 "2006-04-05T18:21:46Z")

</div>

On 4/5/06, Yash [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> If the input string is:  
> Java Ruby ‘Ruby on rails’ “software development” “technology”
> 
> The list of words should be:  
> Java  
> Ruby  
> Ruby on rails  
> software development  
> technology

irb(main):086:0\> s = ‘Java Ruby 'Ruby on rails' “software  
development” “technology”’  
=\> “Java Ruby ‘Ruby on rails’ "software development" "technology"”

irb(main):087:0\> a =  
s.split(/(“._")|('._')|((?=[^”‘])\w+(?=[^"’]))/).find\_all {|s|  
s.match(/\w+/)}  
=\> [“Java”, “Ruby”, “‘Ruby on rails’”, “"software development"  
"technology"”]

If you don’t mind having some array elements that are all whitespace  
you can drop the “find\_all” part.

– James