Forum: Ruby files on a folder too many

Announcement (2017-05-07): is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see and for other Rails- und Ruby-related community platforms.
E05b312ed5c1a4edebb088cae2e15beb?d=identicon&s=25 Mario R. (tcblues)
on 2014-04-23 15:16
I need to perform an action in every single file on a folder that
contains almost a million of files... but when i do:

Dir.foreach(dir) {|f|
#what i want to do in here

it takes forever to start

any idea how to do this
2f375bfe61e8aee50d4c8f21e6bfa5c3?d=identicon&s=25 Durk B. (dirk_b)
on 2014-04-23 18:29
dir -m1 > <files-to-process-with-ruby-in-a-textfile-line-by-line>

Too obvious?
E05b312ed5c1a4edebb088cae2e15beb?d=identicon&s=25 Mario R. (tcblues)
on 2014-04-25 11:55
Same problem, it takes to much, isn't there any way to go through all
the files without getting all the files at once

Durk B. wrote in post #1143875:
> dir -m1 > <files-to-process-with-ruby-in-a-textfile-line-by-line>
> Too obvious?
9627ec27e4fb3a18be02133b4ddaa901?d=identicon&s=25 Joel P. (virtuoso)
on 2014-04-27 22:49
If you have to do something to every file then you're stuck with
iterating through all of them. If you could filter them in some way that
would be helpful, but if you have to perform the action on all of them
you'll have to accept that it will take some time.

You might get minor performance differences by finding ways to get a
list of all the files, but eventually you'll have to start a loop with
almost a million iterations; and it sounds like there's no way to avoid
806d422e42548daa68749f38031ecd40?d=identicon&s=25 Damián M. G. (igorjorobus)
on 2014-04-28 01:02
Did you tried an output like "xxxx files revised" after a file is
revised, to see that Ruby is not stuck, but is working. That way you can
see if Ruby has started or not.
E05b312ed5c1a4edebb088cae2e15beb?d=identicon&s=25 Mario R. (tcblues)
on 2014-04-28 12:51
It seems like every Dir.foreach and Dir.entries(dir) collect all file in
that folder and then continue the execution... so it takes too much
time... my question was if in Ruby I can access to the folders array as
a position in memory and dynamically move that position to the next
item... so it is not necessary to collect all elements from the
Cf8dedaaa16892e8f46bcb51b6ae8117?d=identicon&s=25 Vlad M. (vladm)
on 2014-04-28 13:49
dir ='.')

# Take the first 20 entries and create a map with file names

dir.take(20).flat_map {|item| item.to_s}


# Reads the next entry from dir and returns it as a string. Returns nil
at the end of the stream.

You might have a look into
This topic is locked and can not be replied to.