Slow file access with mapped drives ?!

Ruby
1

Hi,

running ruby 1.8.4 (2006-04-14) [i386-mswin32]
on Windows2000

i have a script that scans some directories for given filename patterns
and does string replacement in those found files.

it works fine, but when it comes to work with files on a mapped drive
the script hangs for about 20secs, afterwards does his work and all OK.

i tried with =

V:/path/to/files
and also with
//servername/c$/path/to/files

and it’s both the same, no difference.

Any ideas how to speed up the file access ?
Are there general problems with unc path and ruby ?

Gilbert

2

Hi,

after some irb it shows the bottleneck lies in
the Dir.glob method

i do something like =

config[‘targetdirs’].each do |dir|

Dir.chdir("//servername/c$/targetdir")
Dir.glob("**/"<<config[‘targetfilepattern’]).each do |file|
…
end
…
end

any alternatives to Dir.glob ?

Regards, Gilbert

3

On 14.05.2007 12:05, Rebhan, Gilbert wrote:

Dir.chdir("//servername/c$/targetdir")
Dir.glob("**/"<<config[‘targetfilepattern’]).each do |file|

Why don’t you just do

Dir.glob("//servername/c$/targetdir/**/"<<config[‘targetfilepattern’]).each
do |file|

Btw, I don’t see any reference to block variable “dir” is this on
purpose?

…
end
…
end

any alternatives to Dir.glob ?

Find.find - but don’t expect that it’s faster because the slowness lies
in file system accesses. Network drives are inherently slower than
local disks.

Kind regards

robert

4

Hi, Robert

5

Hi,

6

On 14.05.2007 14:18, Rebhan, Gilbert wrote:

Why don’t you just do

  • Y:/tempwork
  • T:/rubytest
  • //wvp10175/c$/SCM_Server

??? The code you posted had no dynamic piece there. Did you maybe mean
to write

Dir.chdir("//servername/c$/#{dir}")

…end
That was not in the code you posted.

any alternatives to Dir.glob ?

/*
Find.find - but don’t expect that it’s faster because the slowness lies
in file system accesses. Network drives are inherently slower than
local disks.
*/

it runs in almost half the time, much quicker :slight_smile:

… which might be due to the fact that your new code doesn’t do the
chdir! You changed multiple parameters at the same time so it’s hard to
attribute the perceived performance change to any one parameter.

thanks for the pointer !! , now i have =

config[‘targetdirs’].each do |dir|
Find.find(dir) do |f|
if f =~ /#{config[‘targetfilepattern’]}/
puts "… processing "<<f
filesed!(config[‘replacefrom’],s,f)
end
end
end

Please make sure you post the code you are actually testing - otherwise
everybody else will have a hard time understanding what’s going on let
alone come up with helpful replies.

Regards

robert

7

Hi,

8

wala ba kayong tulog?ang galing

9

On 14.05.2007 14:56, Rebhan, Gilbert wrote:

-----Original Message-----

i don’t want to bother with bad code, but here is the whole thing,
for you to understand what i try to achieve =

I was referring to the fact that you present one bit of code and in your
answer to my posting you present another bit of code as if that was
the original code.

However, there are quite a few things to say about this bit of code.

q.default = “#{ENV[“USERNAME”]}”
q.echo = true}
cvspass = ask("Enter password: ") { |q| q.echo = ‘*’ }
puts “\n\n”

config[‘cvsrepos’].each {|x|
puts “Login CVS Repository >> #{x} …”
IO.popen("#{config[‘CVSEXE’]} -d
:pserver:#{cvsuser}:#{cvspass}@cvsprod:d:/cvsrepos/#{x} login")
}

Why do you use popen if you do not read the pipe? If you use popen
you should make sure the pipe is read from - even if you ignore what you
find because otherwise the other process might get blocked.

  }

Rather use the non block form of gsub unless you want to make sure that
there is no metacharacter interpretation going on. (Even in that case
I’d probably prefer the non block form with a modified replaceString.)

end
tmpFile.close
srcFile .close

Rather use the block form of File.open which is much safer.

s=config[‘replaceto1’]<<"#{@cvsreg}"<<config[‘replaceto2’]

Why do you use “#{@cvsreg}” instead of plain @cvsreg or @cvsreg.to_s?

config[‘targetdirs’].each do |dir|
puts “\n\n”
Find.find(dir) do |f|
if f =~ /#{config[‘targetfilepattern’]}/

You should pull out this regexp compilation from both loops for more
efficiency.

  puts "... processing "<<f
  filesed!(config['replacefrom'],s,f)

Indentation.

  end
end

end
puts “\n\nDone !!”
sleep 1

Why the sleep?

else
puts “\n\nDone !!”
sleep 1

Why the sleep?

  • foo

targetfilepattern: Scm\w+.xml$

replacefrom: “.*”

This pattern will kill you if there are two sections with
in the file. You should at least use the
reluctanct qualifier.

replaceto1: “<![CDATA[”
replaceto2: “]>”

This cries for using a regexp group. Also filesed is pretty inflexible.
I’d rather do something like this:

def file_replace(file, tmp = file + Time.now.usec.to_s)
File.open(tmp, “w”) do |out|
File.open(file) do |inf|
inf.each_line {|line| out.puts(yield(line)) }
end
end
File.mv(tmp, file, :force => true)
end

Now you can do arbitrary line based replacements like

file_replace “foo.txt” do |line|
"# " << line
end

----yaml----

the filesed method is not by me, found it somewhere, don’t remember

Regards, Gilbert

Regards

robert