Hi, folks
the subject has been reported before, basically, ruby File.size() won’t
work properly on windows when file size exceed certain limit (2GB
maybe). One workaround is to require win32/file. Yes, this will get you
correct file, however, this will break the code (in my case, the find
module) … the previous recursive scan of file system stop
functioning. The question is, is there a more “generic” or “elegant”
way of getting correct file size on both *nix and windows?
thanks
-Oliver
[email protected] wrote:
Hi, folks
the subject has been reported before, basically, ruby File.size() won’t
work properly on windows when file size exceed certain limit (2GB
maybe). One workaround is to require win32/file. Yes, this will get you
correct file, however, this will break the code (in my case, the find
module) … the previous recursive scan of file system stop
functioning.
Whoa, what? Can you please elaborate on how win32-file causes problems
with your filesystem scan?
Thanks,
Dan
correction: should be “not working anymore” …
Also, I am not sure why these two modules (or other reasons) are not
working together. Maybe you or someone can enlighten me.
thanks
Oliver
[email protected] wrote:
verbosely.
end
Please define “not working any more”. What isn’t working exactly? Is
there an error message? Or is File.file?(path) not working? If so,
what does “path” look like?
I’m afraid I need more information in order to help you.
Regards,
Dan
Hi,
maybe). One workaround is to require win32/file. Yes, this will get you
correct file, however, this will break the code (in my case, the find
module) … the previous recursive scan of file system stop
functioning. The question is, is there a more “generic” or “elegant”
way of getting correct file size on both *nix and windows?thanks
-Oliver
As the another workaround in windows, try this:
require ‘Win32API’
def File.size(name)
buf = “\0”*36
Win32API.new(‘kernel32’,‘GetFileAttributesEx’,‘PLP’,‘L’).call(name,0,buf)
(buf[28,4].unpack(“L”).first << 32) + buf[32,4].unpack(“L”).first
end
Regards,
Park H.
please see the code segment, once I put in “require win32/file”, the
scanning process is working properly anymore.
require ‘find’
require ‘win32/file’
…
file information
verbosely.
Find.find(options[:dir]) do |path|
if File.file?(path)
if options[:verbose]
puts “Scaning #{path}”
end
size = File.size?(path)
# … do some work on the file
end
end
end
ok, usually, I ran the program and supply directory name “.” as
current, that doesn’t generate any warning message. However, if I
supply a full path, say “C;\temp”, these are the error messages on the
console. This happen as soon as I put in “require win32/file”
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:35:
warning: method rede
ned; discarding old initialize
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:113:
warning: method red
ined; discarding old <=>
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:119:
warning: method red
ined; discarding old blockdev?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:125:
warning: method red
ined; discarding old chardev?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:132:
warning: method red
ined; discarding old executable?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:136:
warning: discarding
ld executable_real?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:141:
warning: method red
ined; discarding old file?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:148:
warning: method red
ined; discarding old ftype
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:168:
warning: method red
ined; discarding old grpowned?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:173:
warning: method red
ined; discarding old owned?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:179:
warning: method red
ined; discarding old pipe?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:183:
warning: discarding
ld socket?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:187:
warning: method red
ined; discarding old readable?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:193:
warning: method red
ined; discarding old readable_real?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:199:
warning: method red
ined; discarding old setgid?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:205:
warning: method red
ined; discarding old setuid?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:212:
warning: method red
ined; discarding old size?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:218:
warning: method red
ined; discarding old sticky?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:224:
warning: method red
ined; discarding old symlink?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:230:
warning: method red
ined; discarding old writable?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:236:
warning: method red
ined; discarding old writable_real?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:242:
warning: method red
ined; discarding old zero?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:262:
warning: method red
ined; discarding old directory?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:335:
warning: method red
ined; discarding old atime
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:341:
warning: method red
ined; discarding old blksize
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:352:
warning: method red
ined; discarding old blocks
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:359:
warning: method red
ined; discarding old ctime
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:366:
warning: method red
ined; discarding old dev
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:376:
warning: method red
ined; discarding old gid
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:382:
warning: method red
ined; discarding old ino
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:392:
warning: method red
ined; discarding old mode
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:398:
warning: method red
ined; discarding old mtime
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:404:
warning: method red
ined; discarding old rdev
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:410:
warning: method red
ined; discarding old nlink
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:416:
warning: method red
ined; discarding old size
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:422:
warning: method red
ined; discarding old uid
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:428:
warning: method red
ined; discarding old inspect
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:451:
warning: method red
ined; discarding old pretty_print
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-0.5.2-mswin32/lib/win32/file.rb:392:
warning: redefine basename
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-0.5.2-mswin32/lib/win32/file.rb:446:
warning: redefine dirname
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-0.5.2-mswin32/lib/win32/file.rb:503:
warning: redefine split
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-0.5.2-mswin32/lib/win32/file.rb:518:
warning: redefine stat
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-0.5.2-mswin32/lib/win32/file.rb:530:
warning: redefine blockdev?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-0.5.2-mswin32/lib/win32/file.rb:537:
warning: redefine chardev?
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-0.5.2-mswin32/lib/win32/file.rb:546:
warning: redefine size
C:\temp
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:263:
warning: instance v
iable @directory not initialized
Hi Park -
thanks for the suggestion. With this redefined function, it works fine
on windows now.
why it is not the case with win32/file, and how can I make it work
cross the platform?
Regards,
-Oliver
Oliver wrote:
ok, usually, I ran the program and supply directory name “.” as
current, that doesn’t generate any warning message. However, if I
supply a full path, say “C;\temp”, these are the error messages on the
console. This happen as soon as I put in “require win32/file”
These are warnins not errors. They are harmless - I redefine a bunch
of methods in the File class.
C:\temp
c:/ruby/lib/ruby/gems/1.8/gems/win32-file-stat-1.2.2-mswin32/lib/win32/file/stat.rb:263:
warning: instance v
iable @directory not initialized
This one is strange, since @directory is initialized. But, I doubt
it’s the source of your problem
None of this helps me, however. You still haven’t told me exactly what
line specifically is failing. With or without win32/file, you are
going to get nil for File.size?(“C:\temp”) or File.size?("."), since
they are directories. It must be something else.
Park H. wrote:
As the another workaround in windows, try this:
require ‘Win32API’
def File.size(name)
buf = “\0”*36
Win32API.new(‘kernel32’,‘GetFileAttributesEx’,‘PLP’,‘L’).call(name,0,buf)
(buf[28,4].unpack(“L”).first << 32) + buf[32,4].unpack(“L”).first
end
I don’t think the problem is File.size. I suspect it’s the “if
File.file?(path)” line failing somewhere it shouldn’t, but the OP isn’t
clear.
Regards,
Dan
Hi Dan -
Find.find(some path) do |path|
puts path
…
end
suppose to loop all the files and sub directories, this is not the case
once import win32/file, the only print out I can get is the very top
level directory I pass in: say, if I run ruby test.rb -d “c:\temp”,
then the only print out is “c:\temp”, all files in that directory are
ignored. It should be pretty easy to verify. It almost seems like find
module has a dependency on File module, and win32/file is in odd with
it.
This is probably as much as I can find at this point …
Oliver
Daniel B. wrote:
I didn’t reallize lstat was implemented on Windows. I’m guessing it’s
an alias, so I’ll need to check the source. But, that’s definitely a
bug in win32-file (or rather, win32-file-stat).I’ll get this fixed and put out a release asap.
Ok, fixed in CVS. I’ll have a release out tonight. If you want the
fix now, just open up win32/file.rb and add this:
def self.lstat(file)
File::Stat.new(file)
end
Regards,
Dan
Dan -
First of all, thanks for fixing this.
A general comment on this issue is: why in Ruby, the basic operation on
files needs special treatment, and short of being cross-platform? If I
am invoking a exotic feature that is windows only, import a win32
module would make more sense to me.
In Python, the operation is uniform cross the platform
import os.path
os.path.getsize(“filename”)
maybe many folks probably won’t agree this, but I tend to regard the
core File.size() won’t work with larger file on windows a bug, though a
minor one.
Best,
Oliver
Oliver wrote:
then the only print out is “c:\temp”, all files in that directory are
ignored. It should be pretty easy to verify. It almost seems like find
module has a dependency on File module, and win32/file is in odd with
it.
Aha! The culprit appears to be lstat. In the find module you’ll see
this line:
if File.lstat(file).directory? then
That returns nil when you include win32-file:
irb(main):001:0> dir = “C:\TMP”
=> “C:\TMP”
irb(main):002:0> File.lstat(dir)
=> #<File::Stat dev=0x2, ino=0, mode=040755, nlink=1, uid=0, gid=0,
rdev=0x2, size=0, blksize=nil, blocks=nil, atime=Thu
Nov 02 09:56:43 -0700 2006, mtime=Fri Oct 21 09:47:09 -0600 2005,
ctime=Fri Oct 21 09:47:09 -0600 2005>
irb(main):003:0> File.lstat(dir).directory?
=> true
irb(main):004:0> require ‘win32/file’
=> true
irb(main):005:0> File.lstat(dir).directory?
=> nil
I didn’t reallize lstat was implemented on Windows. I’m guessing it’s
an alias, so I’ll need to check the source. But, that’s definitely a
bug in win32-file (or rather, win32-file-stat).
I’ll get this fixed and put out a release asap.
Thanks,
Dan
Hi,
win32.c file.
If I’m feeling up to it this weekend, I’ll submit a patch that fixes
File.size and a few of the other methods.Regards,
Dan
The File.size of win32-file still has a bug.
It cannot get correct size when file size is greater than 4Gb(4294967295
byte).
Regards,
Park H.
Oliver wrote:
import os.path
os.path.getsize(“filename”)maybe many folks probably won’t agree this, but I tend to regard the
core File.size() won’t work with larger file on windows a bug, though a
minor one.
Actually, we all agree. It’s a bug in Ruby. It has been brought up,
more than once. The problem is the use of stat instead of stat64 in the
win32.c file.
If I’m feeling up to it this weekend, I’ll submit a patch that fixes
File.size and a few of the other methods.
Regards,
Dan
Park H. wrote:
Actually, we all agree. It’s a bug in Ruby. It has been brought up,
The File.size of win32-file still has a bug.
It cannot get correct size when file size is greater than 4Gb(4294967295
byte).
Looks like I used “L” instead of “Q” in the original buffer. Oops.
I’ll fix that this weekend.
Thanks,
Dan
This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.
Sponsor our Newsletter | Privacy Policy | Terms of Service | Remote Ruby Jobs