I wrote a ruby script to backup files from my hard drive to my flash
drive. For some reason, with one of my flash drives, copying files
with FileUtils.cp_r is about 250 times slower than if I drag and drop
the files from one drive to the other in Windows. I’m suspecting a
buffer size problem. Is there a way to control the buffer size cp_r
selects? If not, what alternatives are there to FileUtils.cp_r? (I
am about to try writing my own, but someone must already have a
solution…)
On 14.01.2009 18:45, Paul L. wrote:
I wrote a ruby script to backup files from my hard drive to my flash
drive. For some reason, with one of my flash drives, copying files
with FileUtils.cp_r is about 250 times slower than if I drag and drop
the files from one drive to the other in Windows. I’m suspecting a
buffer size problem. Is there a way to control the buffer size cp_r
selects? If not, what alternatives are there to FileUtils.cp_r? (I
am about to try writing my own, but someone must already have a
solution…)
Could it be that Windows is doing the copy in the background while cp_r
doesn’t? After all, there is a reason why you must not simply plug off
an USB drive…
Kind regards
robert
On 14.01.2009 18:45, Paul L. wrote:
I wrote a ruby script to backup files from my hard drive to my flash
drive. For some reason, with one of my flash drives, copying files
with FileUtils.cp_r is about 250 times slower than if I drag and drop
the files from one drive to the other in Windows. I’m suspecting a
buffer size problem. Is there a way to control the buffer size cp_r
selects? If not, what alternatives are there to FileUtils.cp_r? (I
am about to try writing my own, but someone must already have a
solution…)
Could it be that Windows is doing the copy in the background while cp_r
doesn’t? After all, there is a reason why you must not simply plug off
an USB drive…
Kind regards
robert
2009/1/17 Paul L. [email protected]:
On Wed, Jan 14, 2009 at 12:45 PM, Paul L. [email protected] wrote:
I wrote a ruby script to backup files from my hard drive to my flash
drive. For some reason, with one of my flash drives, copying files
with FileUtils.cp_r is about 250 times slower than if I drag and drop
the files from one drive to the other in Windows. I’m suspecting a
buffer size problem. Is there a way to control the buffer size cp_r
selects? If not, what alternatives are there to FileUtils.cp_r? (I
am about to try writing my own, but someone must already have a
solution…)
Just another idea: if your files reside on two different physical
disks you might get a benefit from separating reading and writing into
two different threads.
Kind regards
robert
I did some more testing on this. I found that if I ran FileUtils.cp_r
or
FileUtils.cp from within irb, the speed was reasonable (maybe 30s to
copy
25MB to my flash drive.) But, if I double-clicked my script to run it,
it
took about 30 minutes (roughly-- I didn’t wait that long.) I wrote my
own
copy method that opens the files in binary mode and uses a 10 MB buffer
size
(with the read and write methods) and speed dropped back down to 26s.
This
is still 6x slower than Windows drag and drop, but it is tolerable.
Here’s my copy code, in case anyone else has this problem:
def self.copy_file(from_file, to_file)
buffer_size = 10*MB
begin
to_s = File.open(to_file, ‘wb’)
File.open(from_file, ‘rb’) do |from_s|
while(data = from_s.read(buffer_size))
log(“read #{data.length}”)
to_s.write(data)
end
end
rescue
log("Caught error: "+$!)
to_s.close if to_s
raise
end
end
For file backup I’m using rsync on a Linux system, its possible to start
this tool from Ruby.
Rsync copy the difference between the files and is very fast.
I don’t now rsync exist in Windows (maybe Powershell), but its
possible
to run on windows via Cygwin.
Luc.
On Mon, Jan 19, 2009 at 8:33 AM, Robert K.