Forum: Ruby Maximum number of files on a disk ?

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
D0338c0de4cb3c5c17300396159933d1?d=identicon&s=25 Axel Etzold (Guest)
on 2008-11-27 23:22
(Received via mailing list)
Dear all,

is there a way to read out the maximum number of files that can be
created on a hard disk/memory
stick etc., from Ruby ?
I'd need to know this for ext3, NTFS and fat32 systems, and from a
search about ext3, there doesn't
seem to be a general limit, but a number fixed at the creation of the
file system. How could i get that ?

Thank you very much,

Best regards,

Axel
D83785463666ae9bb98a0753eebc8950?d=identicon&s=25 Wayne Vucenic (Guest)
on 2008-11-28 00:16
(Received via mailing list)
On Thu, Nov 27, 2008 at 2:16 PM, Axel Etzold <AEtzold@gmx.de> wrote:

> the maximum number of files that can be created on a hard disk...
> for ... NTFS and fat32 ...

For NTFS, the maximum number of files that can be created is 2**32 - 1.

I'd expect that the number for fat32 is similarly large.

Hope this helps,

Wayne

---
Wayne Vucenic
No Bugs Software
Agile Ruby (but preferably not Rails) Contract Programming in Silicon
Valley since 2001
D0338c0de4cb3c5c17300396159933d1?d=identicon&s=25 Axel Etzold (Guest)
on 2008-11-28 19:32
(Received via mailing list)
-------- Original-Nachricht --------
> Datum: Fri, 28 Nov 2008 08:10:34 +0900
> Von: "Wayne Vucenic" <nightphotos@gmail.com>
> An: ruby-talk@ruby-lang.org
> Betreff: Re: Maximum number of files on a disk ?

>
> Wayne
>
> ---
> Wayne Vucenic
> No Bugs Software
> Agile Ruby (but preferably not Rails) Contract Programming in Silicon
> Valley since 2001

Dear Wayne,

thank you for responding.
I am writing to an NTFS disk, where I am splitting up a large XML file
into smaller
bits, which are saved as individual files ( and closed immediately after
writing to them).
The process runs smoothly until I get the following last correct output

49'000'000   #  <-  number of lines traversed in the XML file so far
(print every 500'000)
<current file title>
1483042     #  <- smaller files written to disk so far

and this error:

lbwiki2.rb:114:in `initialize': Operation not supported - /media/Philips
External Hard Disk/en_wiki/Warnock's\ Dilemma (Errno::EOPNOTSUPP)
  from lbwiki2.rb:114:in `new'
  from lbwiki2.rb:114
  from lbwiki2.rb:85:in `each'
  from lbwiki2.rb:85

In this line 114, there's a File.new for the next small file to be
created. The spaces in the file name are no problem.
I am using Ubuntu 8.10 (32 bit)  + ruby-1.8.7-p72.

I cannot create any files on the disk anymore, neither with Ruby nor on
the command line. This works
again if I delete some file/folder.

So there must be some maximum amount of files that this NTFS disk can
handle, which is way below 2**32-1 ...
What can I do here ?


Thank you very much!

Best regards,

Axel
3afd3e5e05dc9310c89aa5762cc8dd1d?d=identicon&s=25 Tim Hunter (Guest)
on 2008-11-28 19:51
(Received via mailing list)
Axel Etzold wrote:
>> For NTFS, the maximum number of files that can be created is 2**32 - 1.
>> Agile Ruby (but preferably not Rails) Contract Programming in Silicon
> <current file title>
> In this line 114, there's a File.new for the next small file to be created. The spaces 
in the file name are no problem.
> I am using Ubuntu 8.10 (32 bit)  + ruby-1.8.7-p72.
>
> I cannot create any files on the disk anymore, neither with Ruby nor on the command 
line. This works
> again if I delete some file/folder.
>
> So there must be some maximum amount of files that this NTFS disk can handle, which is 
way below 2**32-1 ...
> What can I do here ?
>

I don't know anything about NTFS. I'm just thinking out loud. I wonder
if you've hit some kind of limit on the number of files that can be in
the same directory? Is there some way to organize the files into
multiple directories?
753dcb78b3a3651127665da4bed3c782?d=identicon&s=25 Brian Candler (candlerb)
on 2008-11-28 21:39
Axel Etzold wrote:
> is there a way to read out the maximum number of files that can be
> created on a hard disk/memory
> stick etc., from Ruby ?

Under Linux:  df -i

$ df -i
Filesystem            Inodes   IUsed   IFree IUse% Mounted on
/dev/sda3             786432  246411  540021   32% /

Each file (and directory) takes one inode, so when you've run out of
inodes, you can't create any more files.

> I'd need to know this for ext3, NTFS and fat32 systems

I don't know what limits NTFS and fat32 have. ISTR that the DOS
filesystem had a limit of ~200 files in the root directory, but this
limit didn't apply for subdirectories.

B.
D83785463666ae9bb98a0753eebc8950?d=identicon&s=25 Wayne Vucenic (Guest)
on 2008-11-28 21:45
(Received via mailing list)
On Fri, Nov 28, 2008 at 10:26 AM, Axel Etzold <AEtzold@gmx.de> wrote:
> The spaces in the file name are no problem.

Agreed spaces are no problem, but your file name is somewhat atypical.
Have you tried with a totally plain vanilla file name?

> I cannot create any files ...This works again if I delete some file/folder.

Sorry to ask the obvious, but are you out of space on the disk (or
anywhere close to being out of space)?

Also, Tim makes a good point about files in folders.  Although NTFS has
no limit on this, on older versions of NTFS, things got very slow if
there
were more than a few thousand files in a folder.  That's been fixed in
newer versions, but it's still a good idea not to put lots of files in
the
same folder.

Good luck,

Wayne

---
Wayne Vucenic
No Bugs Software
Agile Ruby (but preferably not Rails) Contract Programming in Silicon
Valley since 2001
This topic is locked and can not be replied to.