SFTP grep on foreach

Ruby
1

Hi guys,

I got

Net::SFTP.start("somehost", "someuser", :password => "somepwd",

:keys => []) do |sftp|
sftp.dir.foreach("/upload/") do |e|

What I want to do is to only parse the files in this given directory
which include “findme” and “foundme” for example.

ls -l | grep “findme|foundme” does the trick in a normal console.

Is there any way to do this via sftp?

Thanks in advance.

Regards

2

Sönke Buhr wrote in post #1016662:

Hi guys,

I got

Net::SFTP.start("somehost", "someuser", :password => "somepwd",

:keys => []) do |sftp|
sftp.dir.foreach("/upload/") do |e|

What I want to do is to only parse the files in this given directory
which include “findme” and “foundme” for example.

ls -l | grep “findme|foundme” does the trick in a normal console.

Is there any way to do this via sftp?

Thanks in advance.

Regards

How about:

base_path = “some/path”

sftp.dir.foreach(base_path) do |entry|
if entry =~ /(found|find)me/
sftp.file.open("#{base_path}/#{entry}, “r”) do |f|
while f.gets
#do something
end
end
end
end

3

7stud – wrote in post #1016702:

Sönke Buhr wrote in post #1016662:

Hi guys,

I got

Net::SFTP.start("somehost", "someuser", :password => "somepwd",

:keys => []) do |sftp|
sftp.dir.foreach("/upload/") do |e|

What I want to do is to only parse the files in this given directory
which include “findme” and “foundme” for example.

ls -l | grep “findme|foundme” does the trick in a normal console.

Is there any way to do this via sftp?

Thanks in advance.

Regards

How about:

base_path = “some/path”

sftp.dir.foreach(base_path) do |entry|
if entry =~ /(found|find)me/
sftp.file.open("#{base_path}/#{entry}, “r”) do |f|
while f.gets
#do something
end
end
end
end

Hi, thanks for your reply.

I kind of do that already.

if %w(my string list).any? {|str| e.name.include? str}

My problem is that there are over 30.000 files on this server so my
intend was to filter unneeded files with the foreach call. I only have
limited access to this server…

Keep it up.

4

On Sun, Aug 14, 2011 at 5:06 PM, Snke B. [email protected] wrote:

Net::SFTP.start(“somehost”, “someuser”, :password => “somepwd”,
:keys => []) do |sftp|
sftp.dir.foreach(“/upload/”) do |e|

What I want to do is to only parse the files in this given directory
which include “findme” and “foundme” for example.

Is there any way to do this via sftp?

Net::SFTP does have a glob method[1]

(with a “scary” disclaimer: “dont expect this method to perform with
the same level of alacrity that ::Dir.glob does; it will work best for
shallow directory hierarchies with relatively few directories, though
it should be able to handle modest numbers of files in each
directory”)

The implementation[2] is very much like 7stud’s suggestion (loop
through all the names and either select or reject) which may the best
option available:

“SCP allows you to use Unix wild cards in the file name when grabbing
a file from the remote machine. SFTP, on the other hand, expects the
full explicit path to the file you want to download.”[3]

The implementation to support globbing in JSch is similar[4]

[1]
http://net-ssh.github.com/sftp/v2/api/classes/Net/SFTP/Operations/Dir.html#M000012
[2]

[3]
http://jessenoller.com/2009/02/05/ssh-programming-with-paramiko-completely-different/
[4]
https://github.com/ePaul/jsch-documentation/blob/master/src/com/jcraft/jsch/ChannelSftp.java#L1694

5

Hey,

glob does the trick. I create two arrays with the wanted filenames and
then combine them and so parse only the wanted files.

Although technically nothing changed (glob just checks for the name as
well) it is much faster now.

Thanks everybody.

6

There’s a tiny wrapper over Net::SFTP that makes it as simple as this

box = Box.new(‘cool_app.com’)

box[‘some/path’].files do |file|
puts file.read if file.name =~ /(found|find)me/
end

It also works with S3 and local FS (and You can copy/move dir/files from
any source to any other)