# Remote file glob

**URL:** <https://www.ruby-forum.com/t/remote-file-glob/74743>\
**Category:** Ruby\
**Created:** [November 7, 2006, 10:54am UTC](https://www.ruby-forum.com/t/remote-file-glob/74743 "2006-11-07T10:54:55Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![greg](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/g/71c47a/32.png) [@greg](https://www.ruby-forum.com/u/greg)\
**Post date:** [November 7, 2006, 10:54am UTC](https://www.ruby-forum.com/t/remote-file-glob/74743/1 "2006-11-07T10:54:55Z")

</div>

What is the fastest way to do a recursive file glob on a remote server?  
Keep in mind that I want to do some filtering based on the directory  
name, file properties, and file extensions.  
(I want to copy these files, but I assume once I have the list, that  
using Net::FTP will be trivial)

I am concerned that just using ftp.chdir and ftp.list will be slow.  
Perhaps there is a faster way using Net:SSH. I had an idea to try to  
run a ruby program on the server (ruby is installed on the remote  
server)  
cmd = "ruby -e ’ #{ File.read( ruby\_glob\_program ) } ’ "

Net::SSH.start(SERVER, :username =\> u, :password =\> p) do |session|  
input, output, error = session.process.popen3( cmd )

---

<div class="post-metadata">

**Author:** ![greg](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/g/71c47a/32.png) [@greg](https://www.ruby-forum.com/u/greg)\
**Post date:** [November 7, 2006, 10:55am UTC](https://www.ruby-forum.com/t/remote-file-glob/74743/2 "2006-11-07T10:55:02Z")

</div>

On 02.11.2006 16:00, greg wrote:

> cmd = "ruby -e ’ #{ File.read( ruby\_glob\_program ) } ’ "  
> That would rather be something like

ruby -e ‘puts Dir[“base/\*\*/\*.txt”]’  
ruby -r find -e ‘Find.find(“base”) {|f| puts f if /.txt$/ =~ f }’

> Net::SSH.start(SERVER, :username =\> u, :password =\> p) do |session|  
> input, output, error = session.process.popen3( cmd )

However, if you just want to find files fast and then copy them, a  
combination of ssh, find, xargs, cp, tar,… might be more efficient.

Kind regards

```
robert
```

---

<div class="post-metadata">

**Author:** ![greg](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/g/71c47a/32.png) [@greg](https://www.ruby-forum.com/u/greg)\
**Post date:** [November 7, 2006, 10:55am UTC](https://www.ruby-forum.com/t/remote-file-glob/74743/3 "2006-11-07T10:55:04Z")

</div>

greg wrote:

> cmd = "ruby -e ’ #{ File.read( ruby\_glob\_program ) } ’ "
> 
> Net::SSH.start(SERVER, :username =\> u, :password =\> p) do |session|  
> input, output, error = session.process.popen3( cmd )

If you’re talking about a Linux or other Unix-like server, indexing a  
filesystem is not something you want to do often. It _is_ slow, even  
using native utilities like “find”, or its cousin “slocate -u”. What  
most sites do is run a cron job in the off hours to build the “slocate  
-u” database, and then do searches using the database.

If you need more current indexing, you probably need to look at the  
application and have it do its own index maintenance.

---

<div class="post-metadata">

**Author:** ![greg](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/g/71c47a/32.png) [@greg](https://www.ruby-forum.com/u/greg)\
**Post date:** [November 7, 2006, 10:55am UTC](https://www.ruby-forum.com/t/remote-file-glob/74743/4 "2006-11-07T10:55:32Z")

</div>

I think I will just use the find command.

I did notice that if I do the following:  
session.process.popen3( cmd ) do |input, output, error|  
puts output.read  
end

For my find command, it will think it is finished reading before find  
is done!  
I solved this by using:

shell = session.shell.sync  
puts shell.exec( find\_cmd ).stdout

Thanks guys!