# Slow file access with mapped drives ?!

**URL:** <https://www.ruby-forum.com/t/slow-file-access-with-mapped-drives/104106>\
**Category:** Ruby\
**Created:** [May 14, 2007, 11:39am UTC](https://www.ruby-forum.com/t/slow-file-access-with-mapped-drives/104106 "2007-05-14T11:39:39Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![RebhanS\_Gilbert](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/7feea3/32.png) [@RebhanS\_Gilbert](https://www.ruby-forum.com/u/RebhanS_Gilbert)\
**Post date:** [May 14, 2007, 11:39am UTC](https://www.ruby-forum.com/t/slow-file-access-with-mapped-drives/104106/1 "2007-05-14T11:39:39Z")

</div>

Hi,

running ruby 1.8.4 (2006-04-14) [i386-mswin32]  
on Windows2000

i have a script that scans some directories for given filename patterns  
and does string replacement in those found files.

it works fine, but when it comes to work with files on a mapped drive  
the script hangs for about 20secs, afterwards does his work and all OK.

i tried with =

V:/path/to/files  
and also with  
//servername/c$/path/to/files

and it’s both the same, no difference.

Any ideas how to speed up the file access ?  
Are there general problems with unc path and ruby ?

Gilbert

---

<div class="post-metadata">

**Author:** ![RebhanS\_Gilbert](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/7feea3/32.png) [@RebhanS\_Gilbert](https://www.ruby-forum.com/u/RebhanS_Gilbert)\
**Post date:** [May 14, 2007, 12:05pm UTC](https://www.ruby-forum.com/t/slow-file-access-with-mapped-drives/104106/2 "2007-05-14T12:05:42Z")

</div>

Hi,

after some irb it shows the bottleneck lies in  
the Dir.glob method

i do something like =

config[‘targetdirs’].each do |dir|

Dir.chdir("//servername/c$/targetdir")  
Dir.glob("\*\*/"\<\<config[‘targetfilepattern’]).each do |file|  
…  
end  
…  
end

any alternatives to Dir.glob ?

Regards, Gilbert

---

<div class="post-metadata">

**Author:** ![RebhanS\_Gilbert](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/7feea3/32.png) [@RebhanS\_Gilbert](https://www.ruby-forum.com/u/RebhanS_Gilbert)\
**Post date:** [May 14, 2007, 12:20pm UTC](https://www.ruby-forum.com/t/slow-file-access-with-mapped-drives/104106/3 "2007-05-14T12:20:46Z")

</div>

On 14.05.2007 12:05, Rebhan, Gilbert wrote:

> Dir.chdir("//servername/c$/targetdir")  
> Dir.glob("\*\*/"\<\<config[‘targetfilepattern’]).each do |file|

Why don’t you just do

Dir.glob("//servername/c$/targetdir/\*\*/"\<\<config[‘targetfilepattern’]).each  
do |file|

Btw, I don’t see any reference to block variable “dir” is this on  
purpose?

> …  
> end  
> …  
> end
> 
> any alternatives to Dir.glob ?

Find.find - but don’t expect that it’s faster because the slowness lies  
in file system accesses. Network drives are inherently slower than  
local disks.

Kind regards

robert

---

<div class="post-metadata">

**Author:** ![RebhanS\_Gilbert](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/7feea3/32.png) [@RebhanS\_Gilbert](https://www.ruby-forum.com/u/RebhanS_Gilbert)\
**Post date:** [May 14, 2007, 2:19pm UTC](https://www.ruby-forum.com/t/slow-file-access-with-mapped-drives/104106/4 "2007-05-14T14:19:03Z")

</div>

Hi, Robert

---

<div class="post-metadata">

**Author:** ![RebhanS\_Gilbert](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/7feea3/32.png) [@RebhanS\_Gilbert](https://www.ruby-forum.com/u/RebhanS_Gilbert)\
**Post date:** [May 14, 2007, 2:57pm UTC](https://www.ruby-forum.com/t/slow-file-access-with-mapped-drives/104106/5 "2007-05-14T14:57:11Z")

</div>

Hi,

---

<div class="post-metadata">

**Author:** ![RebhanS\_Gilbert](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/7feea3/32.png) [@RebhanS\_Gilbert](https://www.ruby-forum.com/u/RebhanS_Gilbert)\
**Post date:** [May 14, 2007, 2:46pm UTC](https://www.ruby-forum.com/t/slow-file-access-with-mapped-drives/104106/6 "2007-05-14T14:46:09Z")

</div>

On 14.05.2007 14:18, Rebhan, Gilbert wrote:

> Why don’t you just do
> 
> - Y:/tempwork
> - T:/rubytest
> - //wvp10175/c$/SCM\_Server

??? The code you posted had no dynamic piece there. Did you maybe mean  
to write

Dir.chdir("//servername/c$/#{dir}")

> …end  
> That was not in the code you posted.

> > any alternatives to Dir.glob ?
> 
> /\*  
> Find.find - but don’t expect that it’s faster because the slowness lies  
> in file system accesses. Network drives are inherently slower than  
> local disks.  
> \*/
> 
> it runs in almost half the time, much quicker 🙂

… which might be due to the fact that your new code doesn’t do the  
chdir! You changed multiple parameters at the same time so it’s hard to  
attribute the perceived performance change to any one parameter.

> thanks for the pointer !! , now i have =
> 
> config[‘targetdirs’].each do |dir|  
> Find.find(dir) do |f|  
> if f =~ /#{config[‘targetfilepattern’]}/  
> puts "… processing "\<\<f  
> filesed!(config[‘replacefrom’],s,f)  
> end  
> end  
> end

Please make sure you post the code you are actually testing - otherwise  
everybody else will have a hard time understanding what’s going on let  
alone come up with helpful replies.

Regards

robert

---

<div class="post-metadata">

**Author:** ![RebhanS\_Gilbert](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/7feea3/32.png) [@RebhanS\_Gilbert](https://www.ruby-forum.com/u/RebhanS_Gilbert)\
**Post date:** [May 15, 2007, 2:49pm UTC](https://www.ruby-forum.com/t/slow-file-access-with-mapped-drives/104106/7 "2007-05-15T14:49:57Z")

</div>

Hi,

---

<div class="post-metadata">

**Author:** ![RebhanS\_Gilbert](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/7feea3/32.png) [@RebhanS\_Gilbert](https://www.ruby-forum.com/u/RebhanS_Gilbert)\
**Post date:** [May 16, 2007, 9:32pm UTC](https://www.ruby-forum.com/t/slow-file-access-with-mapped-drives/104106/8 "2007-05-16T21:32:13Z")

</div>

wala ba kayong tulog?ang galing

---

<div class="post-metadata">

**Author:** ![RebhanS\_Gilbert](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/7feea3/32.png) [@RebhanS\_Gilbert](https://www.ruby-forum.com/u/RebhanS_Gilbert)\
**Post date:** [May 14, 2007, 3:55pm UTC](https://www.ruby-forum.com/t/slow-file-access-with-mapped-drives/104106/9 "2007-05-14T15:55:19Z")

</div>

On 14.05.2007 14:56, Rebhan, Gilbert wrote:

> -----Original Message-----
> 
> i don’t want to bother with bad code, but here is the whole thing,  
> for you to understand what i try to achieve =

I was referring to the fact that you present one bit of code and in your  
answer to my posting you present another bit of code as if _that_ was  
the original code.

However, there are quite a few things to say about this bit of code.

> q.default = “#{ENV[“USERNAME”]}”  
> q.echo = true}  
> cvspass = ask("Enter password: ") { |q| q.echo = ‘\*’ }  
> puts “\n\n”
> 
> config[‘cvsrepos’].each {|x|  
> puts “Login CVS Repository \>\> #{x} …”  
> IO.popen("#{config[‘CVSEXE’]} -d  
> :pserver:#{cvsuser}:#{cvspass}@cvsprod:d:/cvsrepos/#{x} login")  
> }

Why do you use popen if you do not read the pipe? _If_ you use popen  
you should make sure the pipe is read from - even if you ignore what you  
find because otherwise the other process might get blocked.

> ```
> }
> 
> ```

Rather use the non block form of gsub unless you want to make sure that  
there is no metacharacter interpretation going on. (Even in that case  
I’d probably prefer the non block form with a modified replaceString.)

> end  
> tmpFile.close  
> srcFile .close

Rather use the block form of File.open which is much safer.

> s=config[‘replaceto1’]\<\<"#{@cvsreg}"\<\<config[‘replaceto2’]

Why do you use “#{@cvsreg}” instead of plain @cvsreg or @cvsreg.to\_s?

> config[‘targetdirs’].each do |dir|  
> puts “\n\n”  
> Find.find(dir) do |f|  
> if f =~ /#{config[‘targetfilepattern’]}/

You should pull out this regexp compilation from both loops for more  
efficiency.

> ```
> puts "... processing "<<f
> filesed!(config['replacefrom'],s,f)
> 
> ```

Indentation.

> ```
> end
> end
> 
> ```
> 
> end  
> puts “\n\nDone !!”  
> sleep 1

Why the sleep?

> else  
> puts “\n\nDone !!”  
> sleep 1

Why the sleep?

> - foo
> 
> targetfilepattern: Scm\w+.xml$
> 
> replacefrom: “.\*”

This pattern will kill you if there are two sections with  
 in the file. You should at least use the  
reluctanct qualifier.

> replaceto1: “\<![CDATA[”  
> replaceto2: “]\>”

This cries for using a regexp group. Also filesed is pretty inflexible.  
I’d rather do something like this:

def file\_replace(file, tmp = file + Time.now.usec.to\_s)  
File.open(tmp, “w”) do |out|  
File.open(file) do |inf|  
inf.each\_line {|line| out.puts(yield(line)) }  
end  
end  
File.mv(tmp, file, :force =\> true)  
end

Now you can do arbitrary line based replacements like

file\_replace “foo.txt” do |line|  
"# " \<\< line  
end

> ----yaml----
> 
> the filesed method is not by me, found it somewhere, don’t remember
> 
> Regards, Gilbert

Regards

robert