# RE lookahead RE problems

**URL:** <https://www.ruby-forum.com/t/re-lookahead-re-problems/126242>\
**Category:** Rails\
**Created:** [January 16, 2008, 6:08pm UTC](https://www.ruby-forum.com/t/re-lookahead-re-problems/126242 "2008-01-16T18:08:42Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![antonaylward](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/eb8c5e/32.png) [@antonaylward](https://www.ruby-forum.com/u/antonaylward)\
**Post date:** [January 16, 2008, 6:08pm UTC](https://www.ruby-forum.com/t/re-lookahead-re-problems/126242/1 "2008-01-16T18:08:42Z")

</div>

I’ll start by confessing that this comes originally from something I  
worked on in Perl, and I’ve assumed, rightly or wrongly, that regular  
expressions are regular expressions are regular expressions.

See  
[http://www.ilovejackdaniels.com/cheat-sheets/regular-expressions-cheat-sheet/](http://www.ilovejackdaniels.com/cheat-sheets/regular-expressions-cheat-sheet/)

The context is that there are a whole pile of patterns that must be  
preceded by … … well, not words or _some_ punctuation. Call them  
“sort-of zero width”. That is, white space, beginning of line and some  
opening sequences, call the '[‘and (’ and ‘{’ for the sake of the  
example, are allowed.

I’m trying to put the RE into a ‘constant’ so that I don’t have to keep  
repeating it - all the DRY stuff about changes and so forth!

I’m trying to use RE’s lookahead.

This works in perl

```
   $STARTWORD = qr/^|(?<=[\s\(\[\{])/m;

```

There is also the corresponding end word

```
   $ENDWORD = qr/$|(?=[\t\n\,\.\;\:\!\?\)])/om;

```

When I translate these into Ruby I get an error,  
It doesn’t seem to like the lookbehind  
The error message is

```
SyntaxError undefined (?...) sequence: /^|(?<=[\s\(])/

```

Well, possibly. Or it may be that it I’m having problems when combining  
it with an actual pattern.

What I’ve done is separate out the pattern to a constant (and tried to  
eliminate things that might confuse the parser)

STARTWORD = %r{^|(?\<=[\s(])}m

An LO! The parser chokes on that.  
Does it choke because there isn’t actually pattern being compared?  
Well, maybe. If I remove the ‘%r{’ stuff the parser doesn’t choke.  
But it doesn’t choke on

```
ENDWORD = %r{$|(?=[\s,.;:!?)])}m

```

And I seem to be getting confused when combining these with other  
regular expressions because of this inconsistency.

Right now I don’t know if the problem is having the REs as constants.  
Does this make them ‘precompiled’?  
ENDWORD.type ==\> “Regexp”  
so I’m presuming it is. In which case why can’t I precompile STARTWORD?

So: Is it that Ruby can’t handle the ‘?\<=’ lookbehind assertion … or  
what? Am I completely hung up on a wrong track?

–  
Any simple problem can be made insoluble if enough meetings are held to  
discuss it.

---

<div class="post-metadata">

**Author:** ![antonaylward](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/eb8c5e/32.png) [@antonaylward](https://www.ruby-forum.com/u/antonaylward)\
**Post date:** [January 16, 2008, 6:43pm UTC](https://www.ruby-forum.com/t/re-lookahead-re-problems/126242/2 "2008-01-16T18:43:08Z")

</div>

The ruby regular expression engine doesn’t support look-behind.

> **[Comparison of regular expression engines](https://en.wikipedia.org/wiki/Comparison_of_regular_expression_engines)**
>
> This is a comparison of regular expression engines.
> Qt GNU LGPL v. 2.1,
> Qt Commercial
> NOTE: An application using a library for regular expression support does not necessarily support the full set of features of the library, e.g., GNU grep uses PCRE, but supports no lookahead, though PCRE does.

---

<div class="post-metadata">

**Author:** ![antonaylward](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/eb8c5e/32.png) [@antonaylward](https://www.ruby-forum.com/u/antonaylward)\
**Post date:** [January 16, 2008, 6:56pm UTC](https://www.ruby-forum.com/t/re-lookahead-re-problems/126242/3 "2008-01-16T18:56:26Z")

</div>

2008/1/16, Anton J Aylward [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid):

> So: Is it that Ruby can’t handle the ‘?\<=’ lookbehind assertion …

As far as I know, look-behind assertions are not handled by  
Ruby 1.8.\* but I think Oniguruma in 1.9 can.

You should ask your question in Ruby-Talk mailing list,  
which is a better appropriate place for this kind of question.

```
-- Jean-François.

```

---

<div class="post-metadata">

**Author:** ![antonaylward](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/eb8c5e/32.png) [@antonaylward](https://www.ruby-forum.com/u/antonaylward)\
**Post date:** [January 16, 2008, 7:07pm UTC](https://www.ruby-forum.com/t/re-lookahead-re-problems/126242/4 "2008-01-16T19:07:34Z")

</div>

John H. said the following on 16/01/08 12:43 PM:

> The ruby regular expression engine doesn’t support look-behind.
> 
> [Comparison of regular expression engines - Wikipedia](http://en.wikipedia.org/wiki/Comparison_of_regular_expression_engines)

{{ExpletiveDeleted!}}

Suggestions?

–  
The state can’t give you free speech, and the state can’t take it away.  
You’re born with it, like your eyes, like your ears. Freedom is  
something you assume, then you wait for someone to try to take it away.  
The degree to which you resist is the degree to which you are free…  
–Utah Phillips

---

<div class="post-metadata">

**Author:** ![antonaylward](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/eb8c5e/32.png) [@antonaylward](https://www.ruby-forum.com/u/antonaylward)\
**Post date:** [January 16, 2008, 7:22pm UTC](https://www.ruby-forum.com/t/re-lookahead-re-problems/126242/5 "2008-01-16T19:22:50Z")

</div>

Oniguruma

[http://oniguruma.rubyforge.org/](http://oniguruma.rubyforge.org/)

This engine is the RegExp engine for Ruby 1.9 and onwards, so you only  
need  
this gem for 1.8.x.

Jason

---

<div class="post-metadata">

**Author:** ![antonaylward](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/eb8c5e/32.png) [@antonaylward](https://www.ruby-forum.com/u/antonaylward)\
**Post date:** [January 17, 2008, 8:09pm UTC](https://www.ruby-forum.com/t/re-lookahead-re-problems/126242/6 "2008-01-17T20:09:19Z")

</div>

Jason R. said the following on 16/01/08 01:21 PM:

> Oniguruma
> 
> [http://oniguruma.rubyforge.org/](http://oniguruma.rubyforge.org/)
> 
> This engine is the RegExp engine for Ruby 1.9 and onwards, so you only  
> need this gem for 1.8.x.

Roll on 1.9 then, because I get pages and pages of error messages when I  
try installing this gem, starting with

oregexp.c:2:23: error: oniguruma.h: No such file or directory

Now that can’t be because I don’t have the Ruby sources installed, can  
it?

–  
If God does not write LisP, God writes some code so similar to  
LisP as to make no difference.  
See also: [xkcd: Lisp](http://xkcd.com/224/)

---

<div class="post-metadata">

**Author:** ![antonaylward](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/eb8c5e/32.png) [@antonaylward](https://www.ruby-forum.com/u/antonaylward)\
**Post date:** [January 17, 2008, 8:28pm UTC](https://www.ruby-forum.com/t/re-lookahead-re-problems/126242/7 "2008-01-17T20:28:50Z")

</div>

The Oniguruma gem is just a wrapper around the actual library. I haven’t  
installed this myself, though I assumed it would come with the needed  
code.  
You just need to install Oniguruma itself, then get the gem.

Jason

---

<div class="post-metadata">

**Author:** ![antonaylward](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/eb8c5e/32.png) [@antonaylward](https://www.ruby-forum.com/u/antonaylward)\
**Post date:** [February 5, 2008, 8:59pm UTC](https://www.ruby-forum.com/t/re-lookahead-re-problems/126242/8 "2008-02-05T20:59:38Z")

</div>

Thanks for all the help everyone. The problem was solved with the help  
from pullmonkey on Rails Forum! Here is the solution:

Objective:

1. Extract vowels and consonants from a string
2. Handle the conditional treatment of ‘y’ as a vowel under the  
following circumstances:
  - y is a vowel if it is surrounded by consonants
  - y is a consonant if it is adjacent to a vowel

Here is the code that works:

def vowels(name\_str)  
reg = Oniguruma::ORegexp.new(’[aeiou]|(?\<![aeiou])y(?![aeiou])’)  
reg.match\_all(name\_str).to\_s.scan(/./)  
end

def consonants(name\_str)  
reg = Oniguruma::ORegexp.new(’[bcdfghjklmnpqrstvwx]|(?\<=[aeiou])y|  
y(?=[aeiou])’)  
reg.match\_all(name\_str).to\_s.scan(/./)  
end

(Note, the .scan(/./) can be eliminated to return an array)

The major problem was getting the code to accurately treat “y” as a  
consonant. The key to solving this problem was to:

1. define unconditional consonants explicitly (i.e.,  
[bcdfghjklmnpqrstvwx]) – not as [^aeiou] which automatically includes  
“y” thus OVER-RIDING any conditional reatment of “y” that follows

2. define conditional “y” regexp assertions independently, i.e., “| (?  
\<=[aeiou]) y | y (?=[aeiou])” – not “|(?\<=[aeiou]) y (?=[aeiou])”  
which only matches “y” preceded AND followed by a vowel, not preceded  
OR followed by a vowel

HTH.

---

<div class="post-metadata">

**Author:** ![antonaylward](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/eb8c5e/32.png) [@antonaylward](https://www.ruby-forum.com/u/antonaylward)\
**Post date:** [February 4, 2008, 3:49pm UTC](https://www.ruby-forum.com/t/re-lookahead-re-problems/126242/9 "2008-02-04T15:49:07Z")

</div>

The library can be found here:  
[http://www.geocities.jp/kosako3/oniguruma/](http://www.geocities.jp/kosako3/oniguruma/)

I am trying to get look behind working as well. However, having got  
past the errors, I am now wrestling with syntax:

\*\* Starting Rails with development environment…  
Exiting  
/usr/local/lib/ruby/site\_ruby/1.8/rubygems/custom\_require.rb:27:in  
`gem_original_require': ./lib/string_extensions.rb:4: undefined (?...) sequence: /[aeiou]|(?<![aeiou])y(?![aeiou])/ (SyntaxError) ./lib/string_extensions.rb:8: undefined (?...) sequence: /![aeiou]|(? <=[aeiou])y(?=[aeiou])/ from /usr/local/lib/ruby/site_ruby/1.8/ rubygems/custom_require.rb:27:in `require’  
It seems to be complaining about the look-behind and look-ahead  
assertions in the following code fragment (which origuruma is  
supposed  
to support):  
class String  
def vowels  
scan(/[aeiou]|(?\<![aeiou])y(?![aeiou])/i)  
end  
def consonants  
scan(/![aeiou]|(?\<=[aeiou])y(?=[aeiou])/i)  
end  
end  
According to this reference ([サービス終了のお知らせ](http://www.geocities.jp/kosako3/oniguruma/)  
doc/RE.txt), the look behind and look ahead syntax that I am using  
appears to be correct (ref section 7. Extended groups) but apparently  
is not.