# Fast text parsing

**URL:** <https://www.ruby-forum.com/t/fast-text-parsing/74560>\
**Category:** Ruby\
**Created:** [November 7, 2006, 10:48am UTC](https://www.ruby-forum.com/t/fast-text-parsing/74560 "2006-11-07T10:48:11Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![snacktime](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/7feea3/32.png) [@snacktime](https://www.ruby-forum.com/u/snacktime)\
**Post date:** [November 7, 2006, 10:48am UTC](https://www.ruby-forum.com/t/fast-text-parsing/74560/1 "2006-11-07T10:48:11Z")

</div>

Say I’m parsing stuff like http headers, what is going to give better  
performance? Strings with regular expressions? StringIO with  
readline? splitting strings into arrays on a delimiter? Or is it  
going to be so close it’s not really an issue?

Chris

---

<div class="post-metadata">

**Author:** ![snacktime](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/7feea3/32.png) [@snacktime](https://www.ruby-forum.com/u/snacktime)\
**Post date:** [November 7, 2006, 10:48am UTC](https://www.ruby-forum.com/t/fast-text-parsing/74560/2 "2006-11-07T10:48:14Z")

</div>

On Tue, 31 Oct 2006, snacktime wrote:

> Say I’m parsing stuff like http headers, what is going to give better  
> performance? Strings with regular expressions? StringIO with readline?  
> splitting strings into arrays on a delimiter? Or is it going to be so close  
> it’s not really an issue?
> 
> Chris

if you try to write your regular expressions badly enough they can  
surely use  
the most cpu 😉

-a

---

<div class="post-metadata">

**Author:** ![snacktime](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/7feea3/32.png) [@snacktime](https://www.ruby-forum.com/u/snacktime)\
**Post date:** [November 7, 2006, 10:48am UTC](https://www.ruby-forum.com/t/fast-text-parsing/74560/3 "2006-11-07T10:48:17Z")

</div>

On Tue, Oct 31, 2006 at 02:14:42PM +0900, removed\_email\_address@domain.invalid wrote:

> if you try to write your regular expressions badly enough they can surely  
> use  
> the most cpu 😉

It all ‘depends’ 🙂 If you’re doing http header parsing, why not just  
use the header parsing in mongrel. It’s already available as a C  
extension, probably not going to get much faster than that.

But if you want to stick with the strict ruby parsing, experiment and  
see  
what works. I was parsing all the netflix[1] data with ruby for fun and  
I found out some interesting things about text parsing, at least on my  
laptop:

```
- if you only need the data between two delimiter, it was
  faster to do String#index 2x's and slice the data out of the
  middle vs, split and index into the array

- but, if you had 3 items you wanted out, it was faster to do the
  split.

- for simple parsing, regex's were overkill, but if you want to use
  them make sure to compile them once, use them MANY times

```

enjoy,

-jeremy

[1] - [http://www.netflixprize.com/index](http://www.netflixprize.com/index)

---

<div class="post-metadata">

**Author:** ![snacktime](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/7feea3/32.png) [@snacktime](https://www.ruby-forum.com/u/snacktime)\
**Post date:** [November 7, 2006, 10:48am UTC](https://www.ruby-forum.com/t/fast-text-parsing/74560/4 "2006-11-07T10:48:19Z")

</div>

On 10/30/06, Jeremy H. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> > if you try to write your regular expressions badly enough they can surely  
> > use  
> > the most cpu 😉
> 
> It all ‘depends’ 🙂 If you’re doing http header parsing, why not just  
> use the header parsing in mongrel. It’s already available as a C  
> extension, probably not going to get much faster than that.

I am using it actually, but I"m writing a proxy and I need to parse  
the headers the server returns also. I was thinking about just adding  
a parser class to the mongrel parser to do this based on the existing  
one, still not decided though.