Forum: Ruby Help: File.file?, ftype fails oddly

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
2958660cbb27954a8b6acd6c797cb1f8?d=identicon&s=25 Chris (Guest)
on 2006-02-07 18:16
(Received via mailing list)
Hello all,

I've been working on a class+script to crawl through my directories and
thought I had it working just right, when it crashed some 13 minutes
into
running.  I included the stack at the bottom, in case it gives a better
hint to anyone here.

The code where it fails is this method - the if checks fail to see that
filename is a file, and even errors out on the else's
File.ftype(filename)
call:

def get_dir_info_recursive change_into_dir, write_to_file,
put_in_container
Dir.chdir( change_into_dir )
Dir.foreach(".") {|filename|
  if File.directory?(filename) then
    # write_to_file.puts stuff
    # .. recursively call this method
  elsif File.file?(filename) then
    # write_to_file.puts stuff
    # .. simple gather info and add to collection
  else
    puts "ERROR: TYPE=#{File.ftype(filename)} for #{filename} in
#{put_in_container.path}"
  end
}
Dir.chdir('..')
end


My problem solving skills came up with a few alternatives:

1. Maybe the filename/directory path is too long.  After all the path +
filename end up being right about 255 characters long.
    NOPE: Tried starting the loop right in the directory where the file
errors out, even at earlier directories, the error doesn't happen -
the file is correctly identified as a file!

2. Maybe there's a problem with the output file?  It is 11_892_316 bytes
at the error.
    NOPE: Used irb to append another several k to the file with seek
then
putc - no problem there.

3. Is there a hidden runtime problem?
    RAM consumption?  NOPE, I've run the script on larger, though not
deeper directories without it crashing.  Though I would like to know
how to set the RAM usage.  Anyone?

    Stack problems?  NOT purely stack level: simple recursive program
bombs at level 760 with 'stack level too deep'.
    Other stack problems?  Alternating recursion with each loops?
	    Maybe, since 20 directories deep causing 50+ levels in the stack.
And I am making either 1 or 2 new objects per call.

    Ruby + NTFS + WinXP = ?occasional problem?


Any help and ideas are much appreciated.

Thanks,

-Chris

-----------------
Try #1: IF file? ELSE treat it as a directory
Starting file fdata_Q Mon Feb 06 22:40:22 Central Standard Time 2006
drive7.rb:118:in `chdir': Invalid argument -
_EJSRemoteStatelessCrystalItemReconControllerHome_Tie.java
(Errno::EINVAL)
...
         ... 55 levels...
...
Try #2 with drive8: do directory? THEN file? THEN try printing
File.ftype:
C:\CHRIS\ruby\MyFiles>ruby drive8.rb Q
Starting file fdata_Q Mon Feb 06 23:45:27 Central Standard Time 2006
Finished file fdata_Q Mon Feb 06 23:58:37 Central Standard Time 2006
drive8.rb:139:in `ftype': No such file or directory -
_EJSRemoteStatelessCrystalItemReconControllerHome_Tie.java (Errno:
:ENOENT)
        from drive8.rb:139:in `get_dir_info_recursive'
        from drive8.rb:122:in `foreach'
        from drive8.rb:122:in `get_dir_info_recursive'
        from drive8.rb:128:in `get_dir_info_recursive'
        from drive8.rb:122:in `foreach'
        from drive8.rb:122:in `get_dir_info_recursive'
        from drive8.rb:128:in `get_dir_info_recursive'
        from drive8.rb:122:in `foreach'
         ... 54 levels...
        from drive8.rb:122:in `get_dir_info_recursive'
        from drive8.rb:163
        from drive8.rb:148:in `each'
        from drive8.rb:148
430ea1cba106cc65b7687d66e9df4f06?d=identicon&s=25 David Vallner (Guest)
on 2006-02-07 18:31
(Received via mailing list)
DÅ?a Utorok 07 Február 2006 18:15 Chris napísal:
--snip--
>
> 1. Maybe the filename/directory path is too long.  After all the path +
> filename end up being right about 255 characters long.
>     NOPE: Tried starting the loop right in the directory where the file
> errors out, even at earlier directories, the error doesn't happen -
> the file is correctly identified as a file!
>
--snip--
>
>     Ruby + NTFS + WinXP = ?occasional problem?
>

Actually, I think NTFS does have some arbitrary limit on the path
length. Or
the filename length. Or something. That said, I don't think it's quite
related, since you are #chdir-ing around, but who knows.

Try to rework the script to see if you occur it even if using File.find
or
feeding the filenames via lessay Cygwin find, as opposed to recursing
the
directories by hand?

David Vallner

David Vallner
Aee77dba395ece0a04c688b05b07cd63?d=identicon&s=25 Daniel Berger (Guest)
on 2006-02-08 07:24
(Received via mailing list)
Chris wrote:
>
>     puts "ERROR: TYPE=#{File.ftype(filename)} for #{filename} in
> filename end up being right about 255 characters long.
>     NOPE: Tried starting the loop right in the directory where the file
> errors out, even at earlier directories, the error doesn't happen -
> the file is correctly identified as a file!

YEP.  You're hitting the MAX_PATH character limit.  Try the following
sample code to see what I mean:

#initial = "\\\\\?\\C:\\really_long_test_path_to_be_deleted"
initial = "C:\\really_long_test_path_to_be_deleted"
Dir.mkdir(initial)
Dir.chdir(initial)

1.upto(40){ |n|
   dir = "this_is_long_dir_number_#{n}"
   file = "_some_really_long_file_#{n}.bogus"

   File.open(file, "w+"){} # touch
   puts "Created file: #{file}"

   Dir.mkdir(dir)
   puts "Created dir: #{dir}"

   Dir.chdir(dir)
   puts "Changed into dir: #{dir}"

   puts "Current dir length: " + Dir.pwd.length.to_s
   puts
}

And here's the output:

C:\eclipse\workspace\ruby-foo>ruby deepdir.rb
Created file: _some_really_long_file_1.bogus
Created dir: this_is_long_dir_number_1
Changed into dir: this_is_long_dir_number_1
Current dir length: 68

Created file: _some_really_long_file_2.bogus
Created dir: this_is_long_dir_number_2
Changed into dir: this_is_long_dir_number_2
Current dir length: 94

Created file: _some_really_long_file_3.bogus
Created dir: this_is_long_dir_number_3
Changed into dir: this_is_long_dir_number_3
Current dir length: 120

Created file: _some_really_long_file_4.bogus
Created dir: this_is_long_dir_number_4
Changed into dir: this_is_long_dir_number_4
Current dir length: 146

Created file: _some_really_long_file_5.bogus
Created dir: this_is_long_dir_number_5
Changed into dir: this_is_long_dir_number_5
Current dir length: 172

Created file: _some_really_long_file_6.bogus
Created dir: this_is_long_dir_number_6
Changed into dir: this_is_long_dir_number_6
Current dir length: 198

Created file: _some_really_long_file_7.bogus
Created dir: this_is_long_dir_number_7
Changed into dir: this_is_long_dir_number_7
Current dir length: 224

Created file: _some_really_long_file_8.bogus
deepdir.rb:14:in `mkdir': No such file or directory -
this_is_long_dir_number_8 (Errno::ENOENT)
        from deepdir.rb:14
        from deepdir.rb:7:in `upto'
        from deepdir.rb:7

Note that even adding the leading "\\?\" to the directory doesn't seem
to help (though it should).

This isn't the best error message, but that could be Microsoft's fault
- I haven't checked to see what error is returned for this case using C
and GetLastError().

Regards,

Dan
This topic is locked and can not be replied to.