# \# of lines in a file

**URL:** <https://www.ruby-forum.com/t/of-lines-in-a-file/112573>\
**Category:** Ruby\
**Created:** [August 21, 2007, 12:25am UTC](https://www.ruby-forum.com/t/of-lines-in-a-file/112573 "2007-08-21T00:25:39Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![blufur](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/b/d07c76/32.png) [@blufur](https://www.ruby-forum.com/u/blufur)\
**Post date:** [August 21, 2007, 12:25am UTC](https://www.ruby-forum.com/t/of-lines-in-a-file/112573/1 "2007-08-21T00:25:39Z")

</div>

is there built-in method to determine the number of lines in a file?

i tried file.readlines.length but it is very slow (dealing with files

> 1 million lines)

thanks,  
DAN

---

<div class="post-metadata">

**Author:** ![blufur](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/b/d07c76/32.png) [@blufur](https://www.ruby-forum.com/u/blufur)\
**Post date:** [August 21, 2007, 12:35am UTC](https://www.ruby-forum.com/t/of-lines-in-a-file/112573/2 "2007-08-21T00:35:40Z")

</div>

On Tue, Aug 21, 2007 at 07:24:03AM +0900, blufur wrote:

> is there built-in method to determine the number of lines in a file?
> 
> i tried file.readlines.length but it is very slow (dealing with files
> 
> > 1 million lines)

Here are a few alternatives that use less memory than File.readlines  
(which  
slurps in the entire file into memory):

require ‘benchmark’

big\_file = ‘/usr/share/dict/words’

Benchmark.bm do |x|  
x.report(‘streaming’) do  
lines = 0  
File.open(big\_file).each\_line do |line|  
lines += 1  
end  
end

x.report(‘shelling out’) do  
lines = Integer(%x(wc -l ‘#{big\_file}’)[/^\d+/])  
end  
end

On my machine:  
user system total real  
streaming 0.270000 0.010000 0.280000 ( 0.293957)  
shelling out 0.000000 0.000000 0.020000 ( 0.052078)

(The file is 234936 lines.)

marcel

---

<div class="post-metadata">

**Author:** ![blufur](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/b/d07c76/32.png) [@blufur](https://www.ruby-forum.com/u/blufur)\
**Post date:** [August 21, 2007, 12:38am UTC](https://www.ruby-forum.com/t/of-lines-in-a-file/112573/3 "2007-08-21T00:38:40Z")

</div>

On 8/21/07, Jano S. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> There was a thread recently how to process a file as fast as possible  
> – search the archives.

“How to reclaim memory without GC.start”

---

<div class="post-metadata">

**Author:** ![blufur](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/b/d07c76/32.png) [@blufur](https://www.ruby-forum.com/u/blufur)\
**Post date:** [August 23, 2007, 7:50pm UTC](https://www.ruby-forum.com/t/of-lines-in-a-file/112573/4 "2007-08-23T19:50:27Z")

</div>

On Tue, Aug 21, 2007 at 07:34:26AM +0900, Jano S. wrote:

> On 8/21/07, blufur [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:
> 
> > is there built-in method to determine the number of lines in a file?
> > 
> > i tried file.readlines.length but it is very slow (dealing with files
> > 
> > > 1 million lines)
> 
> if on unix:  
> `wc -l #{filename}` or similar (I don’t remember the exact syntax for wc)

Your use of syntax is correct, there. The -l option tells wc to only  
report the number of lines.

---

<div class="post-metadata">

**Author:** ![blufur](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/b/d07c76/32.png) [@blufur](https://www.ruby-forum.com/u/blufur)\
**Post date:** [August 21, 2007, 12:36am UTC](https://www.ruby-forum.com/t/of-lines-in-a-file/112573/5 "2007-08-21T00:36:40Z")

</div>

On 8/21/07, blufur [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> is there built-in method to determine the number of lines in a file?
> 
> i tried file.readlines.length but it is very slow (dealing with files
> 
> > 1 million lines)

if on unix:  
`wc -l #{filename}` or similar (I don’t remember the exact syntax for  
wc)

otherwise:

try counting \r\n or \n. Read file in a loop, counting the occurences.  
There was a thread recently how to process a file as fast as possible  
– search the archives.

---

<div class="post-metadata">

**Author:** ![blufur](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/b/d07c76/32.png) [@blufur](https://www.ruby-forum.com/u/blufur)\
**Post date:** [August 23, 2007, 8:16pm UTC](https://www.ruby-forum.com/t/of-lines-in-a-file/112573/6 "2007-08-23T20:16:31Z")

</div>

On Aug 20, 2007, at 4:33 PM, Marcel Molina Jr. wrote:

> streaming 0.270000 0.010000 0.280000 ( 0.293957)  
> shelling out 0.000000 0.000000 0.020000 ( 0.052078)
> 
> (The file is 234936 lines.)

my attempt:

cfp:~ \> cat a.rb && ruby a.rb Documents/words.txt && wc -l Documents/  
words.txt  
require ‘benchmark’

big\_file = ARGV.shift || ‘/usr/share/dict/words’

Benchmark.bm do |x|  
x.report(‘streaming’) do  
lines = 0  
File.open(big\_file).each\_line do |line|  
lines += 1  
end  
end

x.report(‘shelling out’) do  
lines = Integer(%x(wc -l ‘#{big\_file}’)[/^\d+/])  
end

x.report(‘letting ruby do the counting’) do  
lines = open(big\_file){|fd| fd.each{} and fd.lineno}  
end

x.report(‘wow’) do  
lines = open(big\_file){|fd| fd.read(fd.stat.size).count “\n”}  
end

x.report(‘smart’) do  
class File  
def number\_of\_lines way\_too\_big = 2 \*\* 30  
stat.size \> way\_too\_big ?  
(each{} and lineno) : read(stat.size).count(“\n”)  
end  
end  
lines = open(big\_file){|fd| fd.number\_of\_lines}  
end  
end

```
   user system total real

```

streaming 0.420000 0.010000 0.430000 ( 0.436458)  
shelling out 0.000000 0.000000 0.010000 ( 0.028870)  
letting ruby do the counting 0.290000 0.010000 0.300000  
( 0.296236)  
wow 0.010000 0.010000 0.020000 ( 0.025010)  
smart 0.010000 0.020000 0.030000 ( 0.029373)

483523 Documents/words.txt

a @ [http://drawohara.com/](http://drawohara.com/)

---

<div class="post-metadata">

**Author:** ![blufur](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/b/d07c76/32.png) [@blufur](https://www.ruby-forum.com/u/blufur)\
**Post date:** [August 24, 2007, 10:37am UTC](https://www.ruby-forum.com/t/of-lines-in-a-file/112573/7 "2007-08-24T10:37:11Z")

</div>

From: Ronald F. [[mailto:removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)]

# linecount=`wc -l <#{filename}`.chomp.to\_i

```
 it's ok to lose the #chomp ^^^^^^^

```

kind regards -botp

---

<div class="post-metadata">

**Author:** ![blufur](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/b/d07c76/32.png) [@blufur](https://www.ruby-forum.com/u/blufur)\
**Post date:** [August 24, 2007, 10:23am UTC](https://www.ruby-forum.com/t/of-lines-in-a-file/112573/8 "2007-08-24T10:23:11Z")

</div>

> > if on unix:  
> > `wc -l #{filename}` or similar (I don’t remember the exact  
> > syntax for wc)
> 
> Your use of syntax is correct, there. The -l option tells wc to only  
> report the number of lines.

Nearly correct. It also prints out the filename. A better approach  
when calling from Ruby would be

linecount=`wc -l <#{filename}`.chomp.to\_i