Slow regular-expression engine

Ruby
1

This Awk code takes less than one hundredth of a second to run:

BEGIN {

regex =
“o?o?o?o?o?o?o?o?o?o?o?o?o?o?”
“o?o?o?o?o?o?o?o?o?o?o?o?o?”
“ooooooooooooooooooooooooooooo”

print “ooooooooooooooooooooooooooooo” ~ regex

}

This Ruby code takes 27.5 seconds:

t = Time.now

regex = Regexp.new(
“o?o?o?o?o?o?o?o?o?o?o?o?o?o?” +
“o?o?o?o?o?o?o?o?o?o?o?o?o?” +
“ooooooooooooooooooooooooooooo” )

p “ooooooooooooooooooooooooooooo” =~ regex

puts “#{ Time.now - t } seconds”

See Regular Expression Matching Can Be Simple And Fast

2

On Jul 31, 11:37 am, [email protected] wrote:

print “ooooooooooooooooooooooooooooo” ~ regex
“ooooooooooooooooooooooooooooo” )
Regexp looks and is awful and could be written in a way better way which
takes times of 0.000139965 seconds.

t = Time.now
regex = Regexp.new(/o{0,27}ooooooooooooooooooooooooooooo/ )
p “ooooooooooooooooooooooooooooo” =~ regex
puts “#{ Time.now - t } seconds”

–
Dominik H.
[email protected]

Quoting the article:

… it is possible to write so-called “pathological” regular
expressions that Perl matches very very slowly. In contrast,
there are no regular expressions that are pathological for
the Thompson NFA implementation. Seeing the two graphs side
by side prompts the question, “why doesn’t Perl use the
Thompson NFA approach?” It can, it should …

3

w_a_x_man [email protected] writes:

“ooooooooooooooooooooooooooooo” )

p “ooooooooooooooooooooooooooooo” =~ regex

puts “#{ Time.now - t } seconds”

See Regular Expression Matching Can Be Simple And Fast

In Ruby 1.9 its 1/4 of the time (still slow).

But: One might ask if that is a problem at all, considering that this
Regexp looks and is awful and could be written in a way better way which
takes times of 0.000139965 seconds.

t = Time.now
regex = Regexp.new(/o{0,27}ooooooooooooooooooooooooooooo/ )
p “ooooooooooooooooooooooooooooo” =~ regex
puts “#{ Time.now - t } seconds”

4

On Fri, Jul 31, 2009 at 9:50 AM, w_a_x_man[email protected] wrote:

Quoting the article:

… it is possible to write so-called “pathological” regular
expressions that Perl matches very very slowly. In contrast,
there are no regular expressions that are pathological for
the Thompson NFA implementation. Seeing the two graphs side
by side prompts the question, “why doesn’t Perl use the
Thompson NFA approach?” It can, it should …

So what? I’m sure Ruby core would be happy to consider a patch.

Ben

5

Ben B. wrote:

So what? I’m sure Ruby core would be happy to consider a patch.
Yes we do. And if you could write a time-efficient NFA engine with back
references implemented, please also send that to some scientific
journal,
because that should solve the P=NP problem.

6

On Sat, Aug 1, 2009 at 7:49 AM, Urabe S.[email protected]
wrote:
You should prepare a mail template as inspired by this historic example

--------------------- 8< -----------------------
Dear reader

we have received your proof of Fermat’s Last Theorem. We have to
regret to inform you that there is an error in line ###

--------------------- <8 ----------------------

Cheers
Robert

7

On 31.07.2009 18:49, w_a_x_man wrote:

This Ruby code takes 27.5 seconds:
But: One might ask if that is a problem at all, considering that this
[email protected]

Quoting the article:

… it is possible to write so-called “pathological” regular
expressions that Perl matches very very slowly. In contrast,
there are no regular expressions that are pathological for
the Thompson NFA implementation. Seeing the two graphs side
by side prompts the question, “why doesn’t Perl use the
Thompson NFA approach?” It can, it should …

I am not sure as to what exactly your point is. The awk you have been
using might have used a DFA implementation (sed does so AFAIK).
Nowadays most regular expression engines are NFA because of various
reasons (see “Mastering Regular Expressions”).

Usually NFA’s can be tricked into bad runtime performance with
expressions like the one you wrote. I do not consider this a
disadvantage because on the other hand you can optimize your expression
when working with a NFA, for example by deliberately selecting order of
sub expressions in the expression. And, the expression is really
pathologic, i.e. you would not use something like that in practice.

Btw, why don’t you declare the regular expression directly, i.e.

regex = %r(
o?o?o?o?o?o?o?o?o?o?o?o?o?o?
o?o?o?o?o?o?o?o?o?o?o?o?o?
ooooooooooooooooooooooooooooo
)x

?

Kind regards

robert