Dir.Open hanging on very large directories

Hi Everyone,

I’ve been working on a process to clean up archived files across a
network share but have been running into some difficulty on one specific
folder that contains upwards of 300K files.

When the call Dir.open(folderName) is called, ruby hangs.

Here’s a small stub I wrote that demonstrates the issue:

folderIn = ‘\\server\folder\largedirectory’ #300K+ files

puts “opening dir…”
thisdir = Dir.open(folderIn) if Dir.exists?(folderIn)

puts “opened directory, dir: #{thisdir.to_s}”
puts “r1: #{thisdir.read.to_s}”
puts “r2: #{thisdir.read.to_s}”
puts “r3: #{thisdir.read.to_s}”

running this only produces “Opening dir…”.

Running this on a directory with 5K files produces the intended
behavior:

C:\Users\bergeronc>ruby C:\Users\bergeronc\Desktop\test.rb
opening dir…
opened directory, dir: #Dir:0x25850d0
r1: .
r2: …
r3: Ship2013122508100950.xml

Anyone have any suggestions on how to manage this with such a large
directory? I am using a 32 bit install of ruby running on a 64 bit
2008R2 windows server. Could this be a limitation of the file system or
some other data type within ruby itself?

Please let me know if there is any other information that would be
helpful.

I tried several different ways of creating/opening the directory object
but
none of them seem to ever complete on the 300K file folder.

I also tried this short sub and it also would hang and not finish on the
300K file directory.

folderIn = ‘\\server\folder\5kfiledir’

require ‘win32ole’
fso = WIN32OLE.new(“Scripting.FileSystemObject”)
puts “assign folder dir…”
folder = fso.GetFolder(folderIn)
puts “dir object acquired”

puts “getting file obj”
files = folder.files
puts “got file obj”

puts “file count: #{files.count}” #folder with 300K files hangs here

Apologies if the formatting on this post is off.

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs