# Re: Splitting a sentence with delimiter preserved

**URL:** <https://www.ruby-forum.com/t/re-splitting-a-sentence-with-delimiter-preserved/72540>\
**Category:** Ruby\
**Created:** [October 17, 2006, 10:38pm UTC](https://www.ruby-forum.com/t/re-splitting-a-sentence-with-delimiter-preserved/72540 "2006-10-17T22:38:51Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![phrogz](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/2acd7d/32.png) [@phrogz](https://www.ruby-forum.com/u/phrogz)\
**Post date:** [October 17, 2006, 10:38pm UTC](https://www.ruby-forum.com/t/re-splitting-a-sentence-with-delimiter-preserved/72540/1 "2006-10-17T22:38:51Z")

</div>

From: Ajithkumar Warrier [[mailto:removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)]

> puts b  
> p string.scan( /\w[^.!?]+\S+/ )

#=\> [“This is an example string.”, “The purpose is to save the delimiter  
during split.”, “Does this work.”, “Great!!!.”]

---

<div class="post-metadata">

**Author:** ![phrogz](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/2acd7d/32.png) [@phrogz](https://www.ruby-forum.com/u/phrogz)\
**Post date:** [October 17, 2006, 11:30pm UTC](https://www.ruby-forum.com/t/re-splitting-a-sentence-with-delimiter-preserved/72540/2 "2006-10-17T23:30:55Z")

</div>

On 10/17/06, Gavin K. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:  
That was very quick.

Thank you.

---

<div class="post-metadata">

**Author:** ![phrogz](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/2acd7d/32.png) [@phrogz](https://www.ruby-forum.com/u/phrogz)\
**Post date:** [October 18, 2006, 2:13am UTC](https://www.ruby-forum.com/t/re-splitting-a-sentence-with-delimiter-preserved/72540/3 "2006-10-18T02:13:07Z")

</div>

Hi –

On Wed, 18 Oct 2006, Gavin K. wrote:

> > b = string.split(’#’)  
> > puts b
> 
> p string.scan( /\w[^.!?]+\S+/ )
> 
> #=\> [“This is an example string.”, “The purpose is to save the delimiter  
> during split.”, “Does this work.”, “Great!!!.”]

Also, in 1.9, with oniguruma, you can do:

string.split(/(?\<=.)\s+/)

(negative lookbehind).

David

---

<div class="post-metadata">

**Author:** ![phrogz](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/2acd7d/32.png) [@phrogz](https://www.ruby-forum.com/u/phrogz)\
**Post date:** [October 18, 2006, 4:57am UTC](https://www.ruby-forum.com/t/re-splitting-a-sentence-with-delimiter-preserved/72540/4 "2006-10-18T04:57:24Z")

</div>

[removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> Also, in 1.9, with oniguruma

Is the current (1.8.5) regex engine some (other) well-known engine? For  
example it is very like PCRE, but I take it that it is not PCRE. Just  
curious. m.

---

<div class="post-metadata">

**Author:** ![phrogz](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/2acd7d/32.png) [@phrogz](https://www.ruby-forum.com/u/phrogz)\
**Post date:** [October 18, 2006, 5:05am UTC](https://www.ruby-forum.com/t/re-splitting-a-sentence-with-delimiter-preserved/72540/5 "2006-10-18T05:05:25Z")

</div>

removed\_email\_address@domain.invalid wrote:

> Also, in 1.9, with oniguruma, you can do:
> 
> string.split(/(?\<=.)\s+/)
> 
> (negative lookbehind).

Er, positive lookbehind, I believe you mean.

For completeness, if you wanted to use this form and also wanted to  
allow exclamation points and question marks as sentence delimiters in  
addition to periods, you could use:

string.split( /(?\<=[.!?])\s+/ )

---

<div class="post-metadata">

**Author:** ![phrogz](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/2acd7d/32.png) [@phrogz](https://www.ruby-forum.com/u/phrogz)\
**Post date:** [October 18, 2006, 2:55pm UTC](https://www.ruby-forum.com/t/re-splitting-a-sentence-with-delimiter-preserved/72540/6 "2006-10-18T14:55:54Z")

</div>

On Oct 17, 2006, at 9:55 PM, matt neuburg wrote:

> [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:
> 
> > Also, in 1.9, with oniguruma
> 
> Is the current (1.8.5) regex engine some (other) well-known engine?  
> For  
> example it is very like PCRE, but I take it that it is not PCRE.

Ruby’s current regex engine is pretty limited compared to PCRE or  
Oniguruma. I’m not aware of the name for the current engine.

James Edward G. II

---

<div class="post-metadata">

**Author:** ![phrogz](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/2acd7d/32.png) [@phrogz](https://www.ruby-forum.com/u/phrogz)\
**Post date:** [October 18, 2006, 3:42pm UTC](https://www.ruby-forum.com/t/re-splitting-a-sentence-with-delimiter-preserved/72540/7 "2006-10-18T15:42:23Z")

</div>

Robert K. wrote:

> during split. Does this work. Great!!!. It costs 0.1 dollars."
> 
> > > string.scan( /\w[^.!?]+\S+/ )  
> > > =\> [“This is an example string.”, “The purpose is to save the\ndelimiter  
> > > during split.”, “Does this work.”, “Great!!!.”, “It costs 0.1”, “do  
> > > llars.”]

Down this path leads the madness that is trying to use simple regexp to  
parse something as complex as English grammar. That said, here’s  
another regexp that still works and fixes that particular case:

string = “This is an example string. The purpose is to save the  
delimiter during split. Does this work. Great!!!.”  
string = string + " It costs 0.1 dollars."  
p string.scan( /\w.+?[.!?]+(?=\s|\Z)/ )  
#=\> [“This is an example string.”, “The purpose is to save the  
delimiter during split.”, “Does this work.”, “Great!!!.”, “It costs 0.1  
dollars.”]

It’ll still fail on sentences with embedded quotes that have  
sub-sentences within them.

---

<div class="post-metadata">

**Author:** ![phrogz](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/2acd7d/32.png) [@phrogz](https://www.ruby-forum.com/u/phrogz)\
**Post date:** [October 18, 2006, 10:36am UTC](https://www.ruby-forum.com/t/re-splitting-a-sentence-with-delimiter-preserved/72540/8 "2006-10-18T10:36:01Z")

</div>

On 17.10.2006 22:37, Gavin K. wrote:

> > b = string.split(’#’)  
> > puts b
> 
> p string.scan( /\w[^.!?]+\S+/ )
> 
> #=\> [“This is an example string.”, “The purpose is to save the delimiter  
> during split.”, “Does this work.”, “Great!!!.”]

> > string = “This is an example string. The purpose is to save the  
> > delimiter during split. Does this work. Great!!!.”
> 
> > string = string + " It costs 0.1 dollars."  
> > =\> “This is an example string. The purpose is to save the\ndelimiter  
> > during split. Does this work. Great!!!. It costs 0.1 dollars.”
> 
> > string.scan( /\w[^.!?]+\S+/ )  
> > =\> [“This is an example string.”, “The purpose is to save the\ndelimiter  
> > during split.”, “Does this work.”, “Great!!!.”, “It costs 0.1”, “do  
> > llars.”]

Hm…

```
robert
```