Forum: Ruby Found a neat trick for doing recursive one-liners

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
F0b26d242cc8af969cb83b045d679348?d=identicon&s=25 Gary Watson (Guest)
on 2005-12-27 04:58
(Received via mailing list)
This is probably something everyone in here already knows about, but I
thought it was cool enough that I wanted to post about it.

If you want to create a one liner to say search all the *.txt files
in and under the current directory for text matching "Hello", you can do
this

find -name '*.txt' -exec ruby -ne 'print if /Hello/' '{}' ';'

I know you can do this in pure ruby in like 3 lines if you use the Find
module, but I really wanted to do it with a one liner.  Earlier I tried
something like this

ruby -ne 'print if /Hello/' `find -name '*.txt'`

unfortunately that version would fail if there were any spaces in the
filenames.
E2b3bbf241a5daee3ee7696a7d4b2015?d=identicon&s=25 Vrtwo Lastname (d0t1q)
on 2005-12-27 05:07
(Received via mailing list)
--Cool to use throw ruby into the -exec but I would just use grep in
that scenerio.
42172acdf3c6046f84d644cb0b94642c?d=identicon&s=25 Pat Maddox (pergesu)
on 2005-12-27 05:07
(Received via mailing list)
Or you can use the tools designed for finding stuff :)

find . -name "*.txt" | xargs grep Hello

That version will work for all files.  You can play with find to match
any file you want.

Pat
04a56914cc09f0858d3fca2bf4cbde34?d=identicon&s=25 nobuyoshi nakada (Guest)
on 2005-12-27 05:22
(Received via mailing list)
Hi,

At Tue, 27 Dec 2005 12:57:53 +0900,
Gary Watson wrote in [ruby-talk:172611]:
> I know you can do this in pure ruby in like 3 lines if you use the Find
> module, but I really wanted to do it with a one liner.  Earlier I tried
> something like this
>
> ruby -ne 'print if /Hello/' `find -name '*.txt'`
>
> unfortunately that version would fail if there were any spaces in the
> filenames.

ruby -ne 'BEGIN{ARGV.replace(Dir[ARGV.join("\0")])}; print if /Hello/'
'**/*.txt'
Cb48ca5059faf7409a5ab3745a964696?d=identicon&s=25 unknown (Guest)
on 2005-12-27 05:25
(Received via mailing list)
On Tue, 27 Dec 2005, Gary Watson wrote:

> module, but I really wanted to do it with a one liner.  Earlier I tried
> something like this
>
> ruby -ne 'print if /Hello/' `find -name '*.txt'`

   ruby -e'  puts Dir["**/**"].select{|e| e =~ /a.rb/}  '

-a
F0b26d242cc8af969cb83b045d679348?d=identicon&s=25 Gary Watson (Guest)
on 2005-12-27 05:25
(Received via mailing list)
I apologize for using a brain dead example.  I was more excited about
the
prospect of hitting all files under the current directory recursively,
not
the actual processing I used in my examples.  Thanks for the pointer to
xargs.  I didn't know about that one, I'll have to take a closer look at
it's man page.
E2b3bbf241a5daee3ee7696a7d4b2015?d=identicon&s=25 Vrtwo Lastname (d0t1q)
on 2005-12-27 05:52
(Received via mailing list)
Ara, I always love your examples! :-)  Please keep contributing to the
community!!

BTW, I have the process management class working like a champ.. I'll
share the code with you later!
5fae52bb6ef81a988b84e4cf2b4258e1?d=identicon&s=25 Gary Allum (Guest)
on 2005-12-27 05:55
(Received via mailing list)
I know Im a n00b, but I think more than anything, its good to see you so
excited about Ruby. Learning new things really is fun in Ruby.

On Mon, 26 Dec 2005 20:22:52 -0800, Gary Watson <pfharlock@yahoo.com>
Bc6d88907ce09158581fbb9b469a35a3?d=identicon&s=25 James Britt (Guest)
on 2005-12-27 06:16
(Received via mailing list)
Pat Maddox wrote:
> Or you can use the tools designed for finding stuff :)
>
> find . -name "*.txt" | xargs grep Hello
>
> That version will work for all files.  You can play with find to match
> any file you want.

Assuming you are on a machine with find, xargs, and grep, as opposed to
just Ruby.

I like the idea of assembling command line utils that will work on any
platform where Ruby is installed (e.g., all the machines in my house).

I also like the idea of reinventing the wheel in Ruby because sometimes
you get a better wheel.  Or at least one that is more hackable.

James

--

http://www.ruby-doc.org       - Ruby Help & Documentation
http://www.artima.com/rubycs/ - Ruby Code & Style: Writers wanted
http://www.rubystuff.com      - The Ruby Store for Ruby Stuff
http://www.jamesbritt.com     - Playing with Better Toys
http://www.30secondrule.com   - Building Better Tools
55f28e9c77b35b1539af6be60986b0e4?d=identicon&s=25 Philip Rhoades (Guest)
on 2005-12-27 06:22
(Received via mailing list)
Ara,


On Tue, 2005-12-27 at 13:22 +0900, ara.t.howard@noaa.gov wrote:
> >
> > I know you can do this in pure ruby in like 3 lines if you use the Find
> > module, but I really wanted to do it with a one liner.  Earlier I tried
> > something like this
> >
> > ruby -ne 'print if /Hello/' `find -name '*.txt'`
>
>    ruby -e'  puts Dir["**/**"].select{|e| e =~ /a.rb/}  '


That doesn't seem to do anything . .

Phil.
--
Philip Rhoades

Pricom Pty Limited  (ACN 003 252 275  ABN 91 003 252 275)
GPO Box 3411
Sydney NSW      2001
Australia
Mobile:  +61:(0)411-185-652
Fax:     +61:(0)2-8221-9599
E-mail:  phil@pricom.com.au
1fba4539b6cafe2e60a2916fa184fc2f?d=identicon&s=25 unknown (Guest)
on 2005-12-27 11:37
(Received via mailing list)
Hi --

On Tue, 27 Dec 2005, ara.t.howard@noaa.gov wrote:

>>
>> I know you can do this in pure ruby in like 3 lines if you use the Find
>> module, but I really wanted to do it with a one liner.  Earlier I tried
>> something like this
>>
>> ruby -ne 'print if /Hello/' `find -name '*.txt'`
>
>  ruby -e'  puts Dir["**/**"].select{|e| e =~ /a.rb/}  '

We've strayed a little from the original thing, but along those lines
you could also do:

  ruby -e 'puts Dir["**/**"].grep(/a\.rb/)'

(just guessing about the \. part :-)

or maybe even:

   ruby -e 'puts Dir["**/*a.rb*/"]


David

--
David A. Black
dblack@wobblini.net

"Ruby for Rails", from Manning Publications, coming April 2006!
http://www.manning.com/books/black
86e33dee4a89a8879a26487051c216a8?d=identicon&s=25 Michael Fellinger (Guest)
on 2005-12-27 11:46
(Received via mailing list)
I really prefer the simple variant...
ruby -e 'puts Dir["**/a*.{rb}"]'

finds all .rb that start with 'a', recursive of course :)

Am Dienstag, 27. Dezember 2005 11:36 schrieb dblack@wobblini.net:
33eda546f35b0667856505f21f940529?d=identicon&s=25 Reinder Verlinde (Guest)
on 2005-12-27 11:58
(Received via mailing list)
In article
<810a540e0512262005s3e4c971bl2f2915954c34965f@mail.gmail.com>,
 Pat Maddox <pergesu@gmail.com> wrote:

> Or you can use the tools designed for finding stuff :)
>
> find . -name "*.txt" | xargs grep Hello
>
> That version will work for all files.

<educational type="but not ruby related">
Not quite. For better (maximum?) robustness, pass '-print0' to find and
'-0' to xargs. That will handle filenames with spaces and/or quotes
correctly. (If your filenames have bytes with binary value zero in them,
you still will be out of luck)
</educational>

Reinder
203c2b54209e6e65c50ea2e655a403c4?d=identicon&s=25 W.B.Hill (Guest)
on 2005-12-27 15:48
(Received via mailing list)
On Tue, 27 Dec 2005, Pat Maddox wrote:

> Or you can use the tools designed for finding stuff :)
>
> find . -name "*.txt" | xargs grep Hello
>
> That version will work for all files.  You can play with find to match
> any file you want.

find . -regex '.*\.txt' -print0 | xargs -0 -n1000 grep Hello

-print0 and -0 to avoid trouble with spaces and other metagubbins.
-n1000 for a bit of a speedup
-regex to show you can ;-)
3a5336a2844502fe58c2c06bc12c32ed?d=identicon&s=25 Stefan Walk (Guest)
on 2005-12-27 15:48
(Received via mailing list)
Pat Maddox wrote:
> Or you can use the tools designed for finding stuff :)
>
> find . -name "*.txt" | xargs grep Hello
>
> That version will work for all files.  You can play with find to match
> any file you want.
>
> Pat
>

et@adel:/tmp/rb$ touch 'foo bar.txt'
et@adel:/tmp/rb$ find . -name "*.txt"
../foo bar.txt
et@adel:/tmp/rb$ find . -name "*.txt" | xargs grep Hello
grep: ./foo: No such file or directory
grep: bar.txt: No such file or directory
et@adel:/tmp/rb$ find . -name "*.txt" -print0 | xargs -0 grep Hello
et@adel:/tmp/rb$

Watch out if you are using xargs. It can get pretty nasty, especially if
there is not grep at work, but rm or alike.

Shooting yourself in the foot 101:
$ touch "foo .. bar -rf moo.o"
$ find . -name '*.o' | xargs rm
*BAM*

If you are using find and xargs, always use -print0 and -0,
respectively.

Regards,
Stefan
7264fb16beeea92b89bb42023738259d?d=identicon&s=25 Christian Neukirchen (Guest)
on 2005-12-27 16:33
(Received via mailing list)
Reinder Verlinde <reinder@verlinde.invalid> writes:

> <educational type="but not ruby related">
> Not quite. For better (maximum?) robustness, pass '-print0' to find and
> '-0' to xargs. That will handle filenames with spaces and/or quotes
> correctly. (If your filenames have bytes with binary value zero in them,
> you still will be out of luck)
> </educational>

Know an OS where that is allowed?
203c2b54209e6e65c50ea2e655a403c4?d=identicon&s=25 W.B.Hill (Guest)
on 2005-12-27 16:42
(Received via mailing list)
On Wed, 28 Dec 2005, Christian Neukirchen wrote:

> Reinder Verlinde <reinder@verlinde.invalid> writes:
>
>> <educational type="but not ruby related">
>> Not quite. For better (maximum?) robustness, pass '-print0' to find and
>> '-0' to xargs. That will handle filenames with spaces and/or quotes
>> correctly. (If your filenames have bytes with binary value zero in them,
>> you still will be out of luck)
>> </educational>
>
> Know an OS where that is allowed?

NT 4 kernel mode API is quite happy with \0 in filenames. But it
*really*
confuses the Win32 layer! Haven't played with later versions...
Cb48ca5059faf7409a5ab3745a964696?d=identicon&s=25 unknown (Guest)
on 2005-12-27 17:50
(Received via mailing list)
On Tue, 27 Dec 2005, Philip Rhoades wrote:

>>    ruby -e'  puts Dir["**/**"].select{|e| e =~ /a.rb/}  '
>
>
> That doesn't seem to do anything . .

then you probably don't have any files named 'a.rb' under the current
directory - i seem to have several hundred ;-)

cheers.

-a
1fba4539b6cafe2e60a2916fa184fc2f?d=identicon&s=25 unknown (Guest)
on 2005-12-27 18:36
(Received via mailing list)
Hi --

On Wed, 28 Dec 2005, ara.t.howard@noaa.gov wrote:

> On Tue, 27 Dec 2005, Philip Rhoades wrote:
>
>>>    ruby -e'  puts Dir["**/**"].select{|e| e =~ /a.rb/}  '
>>
>>
>> That doesn't seem to do anything . .
>
> then you probably don't have any files named 'a.rb' under the current
> directory - i seem to have several hundred ;-)

Or abrb, or acrb, or airbag, or....  :-)


David

--
David A. Black
dblack@wobblini.net

"Ruby for Rails", from Manning Publications, coming April 2006!
http://www.manning.com/books/black
Cb48ca5059faf7409a5ab3745a964696?d=identicon&s=25 unknown (Guest)
on 2005-12-27 18:39
(Received via mailing list)
On Wed, 28 Dec 2005 dblack@wobblini.net wrote:

>>
>> then you probably don't have any files named 'a.rb' under the current
>> directory - i seem to have several hundred ;-)
>
> Or abrb, or acrb, or airbag, or....  :-)

indeed.  or diectories.  that's the nice thing about using select:

   ruby -e'  puts Dir["**/**"].select{|e| test ?f, e and e =~ /^a\.rb$/}
'

-a
1fba4539b6cafe2e60a2916fa184fc2f?d=identicon&s=25 unknown (Guest)
on 2005-12-27 18:51
(Received via mailing list)
On Wed, 28 Dec 2005, ara.t.howard@noaa.gov wrote:

>>>>
>>>> That doesn't seem to do anything . .
>>>
>>> then you probably don't have any files named 'a.rb' under the current
>>> directory - i seem to have several hundred ;-)
>>
>> Or abrb, or acrb, or airbag, or....  :-)
>
> indeed.  or diectories.  that's the nice thing about using select:
>
>  ruby -e'  puts Dir["**/**"].select{|e| test ?f, e and e =~ /^a\.rb$/}  '

I was just obliquely pointing out the lack of ^ and \. and $ :-)  But
it's true that Dir.[] is not very seletcive....  In fact, I think I
once submitted an RCR to let it take a second argument that would be
tested for (like: Dir["**/**"],?f]) but it was rejected.  So here I am
several years later still acting as if Dir.[] could read my mind....


David

--
David A. Black
dblack@wobblini.net

"Ruby for Rails", from Manning Publications, coming April 2006!
http://www.manning.com/books/black
7264fb16beeea92b89bb42023738259d?d=identicon&s=25 Christian Neukirchen (Guest)
on 2005-12-27 19:03
(Received via mailing list)
"W.B.Hill" <mail@wbh.org> writes:

>>
>> Know an OS where that is allowed?
>
> NT 4 kernel mode API is quite happy with \0 in filenames. But it
> *really* confuses the Win32 layer! Haven't played with later
> versions...

So you need to pass the filename size all over?  Ugh. :P
7da0c2cbd3e9a596006b994b6a36f09c?d=identicon&s=25 Daniel Calvelo (Guest)
on 2005-12-27 22:03
(Received via mailing list)
Great!

Would any mad enough rubyists start the equivalent of
http://ppt.perl.org/? For the impatient, "perl power tools: unix
reconstruction project" is an on-going attempt at writing (most of) the
BSD command set in Perl.

Not quite as mad as BASIC in TeX, but potentially useful indeed.

--DCA
This topic is locked and can not be replied to.