# Correct way to process a file in Ruby?

**URL:** <https://www.ruby-forum.com/t/correct-way-to-process-a-file-in-ruby/83899>\
**Category:** Ruby\
**Created:** [January 30, 2007, 6:28pm UTC](https://www.ruby-forum.com/t/correct-way-to-process-a-file-in-ruby/83899 "2007-01-30T18:28:03Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![daveh](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/85e7bf/32.png) [@daveh](https://www.ruby-forum.com/u/daveh)\
**Post date:** [January 30, 2007, 6:28pm UTC](https://www.ruby-forum.com/t/correct-way-to-process-a-file-in-ruby/83899/1 "2007-01-30T18:28:03Z")

</div>

Hi

What is the correct way to code the following in Ruby?

open file  
read field from file  
determine from field the type of file  
case file type in  
type 1) do this (lots of stuff)  
type 2) do that (even more stuff)  
type 3) do something else (you get the picture)  
\*) error  
esac  
close file

I presently have this coded as one big method but the number of cases  
are large and the complexity of the processing increasing. This is going  
to end up a hell of a larger method.

I then thought about breaking it into a number of methods but that  
resulted in opening and closing the file a number of times (in each  
method) which feels bad

Would it be right to use a global variable as a file descriptor and open  
the file in one method which returns a fd and then having a number of  
methods processing using the fd and then finally having a method that  
closes the file?

Sorry if I’m seem to be grasping for the correct Ruby language to  
express this - I’m very new to Ruby coming from a C and shell  
background.

Daveh

---

<div class="post-metadata">

**Author:** ![daveh](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/85e7bf/32.png) [@daveh](https://www.ruby-forum.com/u/daveh)\
**Post date:** [January 30, 2007, 6:35pm UTC](https://www.ruby-forum.com/t/correct-way-to-process-a-file-in-ruby/83899/2 "2007-01-30T18:35:25Z")

</div>

Recently coded something similar.  
I just slurp the file into an array (File.readlines) and then  
split the array into separate arrays of each record type.

cheers  
Chris

---

<div class="post-metadata">

**Author:** ![daveh](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/85e7bf/32.png) [@daveh](https://www.ruby-forum.com/u/daveh)\
**Post date:** [January 30, 2007, 6:40pm UTC](https://www.ruby-forum.com/t/correct-way-to-process-a-file-in-ruby/83899/3 "2007-01-30T18:40:23Z")

</div>

Hi Chris,

> I just slurp the file into an array (File.readlines) and then  
> split the array into separate arrays of each record type.

I guess I should have mentioned that these files are binary format and  
can be quite large - anywhere from 3M to 50MB, so I was hoping to  
process them on disk.

Would reading into an array still be appropriate in Ruby?

---

<div class="post-metadata">

**Author:** ![daveh](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/85e7bf/32.png) [@daveh](https://www.ruby-forum.com/u/daveh)\
**Post date:** [January 30, 2007, 6:49pm UTC](https://www.ruby-forum.com/t/correct-way-to-process-a-file-in-ruby/83899/4 "2007-01-30T18:49:29Z")

</div>

On 1/30/07, Dave H. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> 

If you are on a Unix system, look into the Ruby mmap library.

[http://raa.ruby-lang.org/project/mmap/0.2.6](http://raa.ruby-lang.org/project/mmap/0.2.6)

This will only read in the parts of the file you need. There is a  
similar library for the windows platform (but I have not used that  
one).

My solution to handling stuff like this is to create a parser class.

class FooParser

def initialize( filename )  
@mmap = Mmap.new(filename, ‘r’)  
end

def close  
return if @mmap.nil?  
@mmap.unmap  
@mmap = nil  
end

def parse\_info\_type1  
# do stuff here to parse one type of information from the mmap  
object  
end

def parse\_info\_type2  
# etc …  
end

end

It works very well, and mmap allows me to handle gigabyte sized files  
without hogging all the system memory.

Blessings,  
TwP

---

<div class="post-metadata">

**Author:** ![daveh](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/85e7bf/32.png) [@daveh](https://www.ruby-forum.com/u/daveh)\
**Post date:** [January 30, 2007, 6:49pm UTC](https://www.ruby-forum.com/t/correct-way-to-process-a-file-in-ruby/83899/5 "2007-01-30T18:49:57Z")

</div>

Dave H. wrote:

> What is the correct way to code the following in Ruby?
> 
> open file  
> read field from file  
> determine from field the type of file  
> case file type in  
> type 1) do this (lots of stuff)  
> type 2) do that (even more stuff)  
> type 3) do something else (you get the picture)  
> \*) error  
> esac  
> close file

You shouldn’t need to open/close the file more than once. Unless I’m  
misunderstanding something, you should be able to just do something like  
this:

File.open(‘huge.txt’) do |file|  
first\_line = file.gets  
file\_type = determine\_file\_type(first\_line)  
case file\_type  
when ‘csv’: process\_csv(file)  
when ‘tab\_delimited’: process\_tab(file)  
else  
puts “Error! OMG!”  
end  
end

def process\_csv(file)  
file.each do |line|  
# do something with the line  
end  
end

The main File.open block handles the opening and closing of the file,  
and you just pass the file handle to the methods that do the actual  
processing. Nice and simple.