# Trouble reading a set of files recursively

**URL:** <https://www.ruby-forum.com/t/trouble-reading-a-set-of-files-recursively/165984>\
**Category:** Ruby\
**Created:** [April 20, 2009, 4:51pm UTC](https://www.ruby-forum.com/t/trouble-reading-a-set-of-files-recursively/165984 "2009-04-20T16:51:01Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![casper\_the\_ghost](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/58956e/32.png) [@casper\_the\_ghost](https://www.ruby-forum.com/u/casper_the_ghost)\
**Post date:** [April 20, 2009, 4:51pm UTC](https://www.ruby-forum.com/t/trouble-reading-a-set-of-files-recursively/165984/1 "2009-04-20T16:51:01Z")

</div>

Hello,  
I am doing a batch run, where I have several .csv files with the same  
name in different sub-directories. At the end of the batch run, I want  
to cat all the individual files together, and load them into the  
database.

Here is the code:  
Dir["\*\*/store\_need\*"].each do |path|  
single\_file\_nm = path.to\_s  
single\_file = File.open("#{single\_file\_nm}", “r”)  
single\_file.each do |line|  
store\_need\_file \<\< line  
puts line  
end  
single\_file.close  
end

I am recursively looking for the store\_need.csv files, and then  
reading them into a single store\_need\_file. The problem is that this  
code is not reading the file at all. I can see that the path is  
defined correctly.

Any help is appreciated. I am running this on windows XP.

Kaushik

---

<div class="post-metadata">

**Author:** ![casper\_the\_ghost](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/58956e/32.png) [@casper\_the\_ghost](https://www.ruby-forum.com/u/casper_the_ghost)\
**Post date:** [April 20, 2009, 5:04pm UTC](https://www.ruby-forum.com/t/trouble-reading-a-set-of-files-recursively/165984/2 "2009-04-20T17:04:30Z")

</div>

removed\_email\_address@domain.invalid wrote:

> Here is the code:  
> Dir["\*\*/store\_need\*"].each do |path|  
> single\_file\_nm = path.to\_s  
> single\_file = File.open("#{single\_file\_nm}", “r”)  
> single\_file.each do |line|  
> store\_need\_file \<\< line  
> puts line  
> end  
> single\_file.close  
> end

I just tried your code on a Linux box, and it ran as it should. I don’t  
do Windows, but at least this tells you to focus on OS specific issues.  
Could it have something to do with drive letters or slashes instead of  
backslashes?

---

<div class="post-metadata">

**Author:** ![casper\_the\_ghost](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/58956e/32.png) [@casper\_the\_ghost](https://www.ruby-forum.com/u/casper_the_ghost)\
**Post date:** [April 20, 2009, 5:33pm UTC](https://www.ruby-forum.com/t/trouble-reading-a-set-of-files-recursively/165984/3 "2009-04-20T17:33:50Z")

</div>

2009/4/20 removed\_email\_address@domain.invalid [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid):

> I am doing a batch run, where I have several .csv files with the same  
> name in different sub-directories. At the end of the batch run, I want  
> to cat all the individual files together, and load them into the  
> database.
> 
> Here is the code:  
> Dir[“\*\*/store\_need\*”].each do |path|  
> single\_file\_nm = path.to\_s

Why do you convert a string to a string and then do string  
interpolation in the next line?

> ```
> single_file = File.open("#{single_file_nm}", "r")
> 
> ```

The string interpolation is superfluous plus you should rather use the  
block form of File.open.

> ```
> single_file.each do |line|
> store_need_file << line
> 
> ```

Where do you open store\_need\_file?

> ```
> puts line
> end
> single_file.close
> 
> ```
> 
> end
> 
> I am recursively looking for the store\_need.csv files, and then  
> reading them into a single store\_need\_file. The problem is that this  
> code is not reading the file at all. I can see that the path is  
> defined correctly.

This is a shorter variant if your individual CSV files are small:

File.open “load.csv”, “w” do |io|  
Dir[“\*\*/store\_need\*”].each do |path|  
# note: puts instead of write to get the terminating newline  
io.puts(File.read(path))  
end  
end

If your individual files are large I would probably not bother to  
combine them. If you still want to do it then I would do something  
like this

# untested

NL = “\r\n”.freeze  
BL = 1024

File.open “load.csv”, “wb” do |fout|  
buffer = “”

Dir[“\*\*/store\_need\*”].each do |path|  
File.open path, “rb” do |fin|  
while fin.read(BL, buffer)  
fout.write(buffer)  
end  
end

```
# only needed if files are not terminated properly:
# fout.write(NL)

```

end  
end

Cheers

robert