# Regexing a file's contents without reading the whole thing?

**URL:** <https://www.ruby-forum.com/t/regexing-a-files-contents-without-reading-the-whole-thing/179628>\
**Category:** Ruby\
**Created:** [November 30, 2009, 9:32pm UTC](https://www.ruby-forum.com/t/regexing-a-files-contents-without-reading-the-whole-thing/179628 "2009-11-30T21:32:07Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![rogerdpack](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/45deac/32.png) [@rogerdpack](https://www.ruby-forum.com/u/rogerdpack)\
**Post date:** [November 30, 2009, 9:32pm UTC](https://www.ruby-forum.com/t/regexing-a-files-contents-without-reading-the-whole-thing/179628/1 "2009-11-30T21:32:07Z")

</div>

I see that it is possible currently to parse through a file without  
reading the whole thing into RAM, a la

a = File.open(‘a’, ‘r’)  
a.lines{|line|  
if line =~ /some regex/  
…  
end  
}

But what if I can to do something like  
a = File.read(‘a’).scan /some regex/

is that possible?

Thanks.  
-r

---

<div class="post-metadata">

**Author:** ![rogerdpack](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/45deac/32.png) [@rogerdpack](https://www.ruby-forum.com/u/rogerdpack)\
**Post date:** [November 30, 2009, 9:55pm UTC](https://www.ruby-forum.com/t/regexing-a-files-contents-without-reading-the-whole-thing/179628/2 "2009-11-30T21:55:22Z")

</div>

Roger P. wrote:

> But what if I can to do something like  
> a = File.read(‘a’).scan /some regex/
> 
> is that possible?
> 
> Thanks.  
> -r

File.open(’/usr/share/dict/words’).grep /ruby/i

---

<div class="post-metadata">

**Author:** ![rogerdpack](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/45deac/32.png) [@rogerdpack](https://www.ruby-forum.com/u/rogerdpack)\
**Post date:** [December 1, 2009, 1:59pm UTC](https://www.ruby-forum.com/t/regexing-a-files-contents-without-reading-the-whole-thing/179628/3 "2009-12-01T13:59:56Z")

</div>

2009/11/30 Roger P. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid):

> But what if I can to do something like  
> a = File.read(‘a’).scan /some regex/
> 
> is that possible?

If you know that matches will never cross line breaks you can do

a = []  
File.foreach(“a”) do |line|  
line.scan /regex/ do |m|  
a \<\< m  
end

# alternative:

a.concat(line.scan(/regex/))  
end

If matches can cross line breaks the whole store becomes more  
complicated and your solution with File.read is probably the simplest  
way to do it (if files aren’t too large).

Kind regards

robert

---

<div class="post-metadata">

**Author:** ![rogerdpack](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/45deac/32.png) [@rogerdpack](https://www.ruby-forum.com/u/rogerdpack)\
**Post date:** [December 2, 2009, 2:33am UTC](https://www.ruby-forum.com/t/regexing-a-files-contents-without-reading-the-whole-thing/179628/4 "2009-12-02T02:33:45Z")

</div>

On 11/30/09, Roger P. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> But what if I can to do something like  
> a = File.read(‘a’).scan /some regex/
> 
> is that possible?

The library which makes this possible is sequence. I’m coding this  
from memory, so I’m likely to get something wrong, but the equivalent  
in sequence looks more or less like this:

require ‘rubygems’  
require ‘sequence’  
require ‘sequence/file’

seq=Sequence.new(File.open(‘a’))  
seq.scan\_until(/some regex/)

Keep the following in mind:

1. Sequence#scan works like StringScanner#scan, not String#scan.
2. The pattern to be matched must have a max length (4k by default, I  
think; it can be changed).
3. If your pattern is guaranteed to not contain a nl, you’re better  
off with readline, as robert said.