Forum: Ruby State of the union for Ruby CLI libraries?

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
4bcdbf252c83c459de4f29f697c5760d?d=identicon&s=25 John Feminella (Guest)
on 2010-06-17 16:22
(Received via mailing list)
I am starting construction on a somewhat complicated internal
application whose primary interface will be the command line. In the
past, for similar projects, I've just used OptionParser, but I think I
need something a little beefier. Specifically, I think I need these
features:

* colorized output
* sensible option parsing
* sensible enumeration of options for help output
* support for `git`-like subcommands, e.g., `app create`, `app
delete`, etc., that can be intuitively and cleanly mapped to methods
* ability to be insensitive to option ordering, e.g. `app create --foo
quux --bar` should be semantically identical to `app create --bar
--foo quux`

Does that sort of thing exist? I've noticed there are some apparently
more powerful alternatives like Thor, which is more of a full-stack
scripting framework, and I'm wondering what else might be useful to
take a look at.

Any and all suggestions would be much appreciated. Thanks!
45196398e9685000d195ec626d477f0e?d=identicon&s=25 Thomas Sawyer (7rans)
on 2010-06-17 16:22
(Received via mailing list)
On Jun 13, 12:07 pm, John Feminella <jo...@distb.net> wrote:
> delete`, etc., that can be intuitively and cleanly mapped to methods
> * ability to be insensitive to option ordering, e.g. `app create --foo
> quux --bar` should be semantically identical to `app create --bar
> --foo quux`
>
> Does that sort of thing exist? I've noticed there are some apparently
> more powerful alternatives like Thor, which is more of a full-stack
> scripting framework, and I'm wondering what else might be useful to
> take a look at.
>
> Any and all suggestions would be much appreciated. Thanks!

OptionParser generally works fine --even for the items you mention. To
handle subcommands just pop off the top of ARGV before parsing (or for
more advanced use, find the first non-option item). The order of
options is not an issue, and help is pretty easy since OptionPaser
responds to #to_s. Running that through a colorizing filter shouldn't
be too hard. For color try http://rubyworks.github.com/ansi.

There are other options out there though, check out HighLine and
Oyster for instance.
F30b04ed4d0b3fc4bc791a28815f34ca?d=identicon&s=25 Rein Henrichs (reinh)
on 2010-06-17 16:22
(Received via mailing list)
On 2010-06-13 09:07:31 -0700, John Feminella said:

> delete`, etc., that can be intuitively and cleanly mapped to methods
> * ability to be insensitive to option ordering, e.g. `app create --foo
> quux --bar` should be semantically identical to `app create --bar
> --foo quux`
>
> Does that sort of thing exist? I've noticed there are some apparently
> more powerful alternatives like Thor, which is more of a full-stack
> scripting framework, and I'm wondering what else might be useful to
> take a look at.
>
> Any and all suggestions would be much appreciated. Thanks!

After recently reviewing the state of the art for Ruby CLI tools myself
as we moved Puppet to a sub-command-based executable structure, I
couldn't find a single, cohesive library that provides all of these
features. The domain is a full of NIH and wheel reinvention. There also
doesn't seem to be much thought given to combining multiple tools in a
unix-like fashion (small tools chained together to provide more
complete functionality). They all seem to want to do a bit of
everything but not enough of anything.

OptionParser is convenient because it's available in the standard
library. There are a number of other option parsing tools but they
(imo) don't provide significant benefit to offset the cost of an
additional dependency.

As far as subcommands go, there is rake (which can be used as a
stand-alone application DSL), thor (by Yehuda Katz), main (by Ara
Howard), the Command code that is a part of the github gem (by Chris
Wanstrath), the Application code that is a part of Puppet and I'm sure
at least a dozen other reimplementations hidden in various gems and
libraries.

I found Rake, Thor and Main gave me too much friction for the type of
CLI tool I was interested in. Option parsing DSLs that were cumbersome,
cryptic error handling and unweildy API design for my use case. That
said, please try them and see if they work for you.

There are probably another dozen tools I haven't found yet or forgot
didn't mention. Github queries for "Ruby CLI" and similar return a
staggering number of results. I'd welcome any suggestions for my own
use as well.

Finally I recently wrote a very small sub-command DSL gem[1] in a fit
of my own NIH. It almost certainly doesn't do what you wantbut you are
welcome to try it. The design goal was to do one thing (subcommands)
and do it well, and to get out of your way and let you use your own
tools for everything else. I would welcome feedback.

[1] http://github.com/reinh/commandant
753dcb78b3a3651127665da4bed3c782?d=identicon&s=25 Brian Candler (candlerb)
on 2010-06-18 10:13
Rein Henrichs wrote:
> There are probably another dozen tools I haven't found yet or forgot
> didn't mention.

There's one I can never remember the name of, but is something to do
with prostitutes. Oh yes, it's "trollop".

Examples at http://trollop.rubyforge.org/ suggest that subcommands are
supported.
0026dd77fd9ecc97b36e5b79cdbcf590?d=identicon&s=25 R. Kumar (sentinel)
on 2010-06-18 11:17
John Feminella wrote:

>
> Does that sort of thing exist? I've noticed there are some apparently
> more powerful alternatives like Thor, which is more of a full-stack
> scripting framework, and I'm wondering what else might be useful to
> take a look at.
>
> Any and all suggestions would be much appreciated. Thanks!

There's also visionmedia's commander
http://github.com/visionmedia/commander.

I've played with it a bit, but not started working on it. I've started
converting some shell-scripts to ruby, and i've rolled my first one
using optionparser.

I may try commander for the second one. But I am afraid of a large code
base, and it getting broken at some stage. For the ease of some option
parsing and a few more tiny goodies, I can;t have my entire app
dependent on some large code base.

My current apps manipulate files largely using "sed", "grep", "cut" and
some other unix commands. So the meat of my programs is still not served
by these frameworks.

I once tried Thor and liked it. But its written by Y Katz who is into
Rails currently. So i don't know if it's being maintained (if you run
into issues). I;ve been bitten earlier using a gem written by a high
profile programmer who was too involved elsewhere to fix a bug in his
gem or even reply. :-(
0026dd77fd9ecc97b36e5b79cdbcf590?d=identicon&s=25 R. Kumar (sentinel)
on 2010-06-18 11:22
John Feminella wrote:

>
> * colorized output

That's a trivial issue. All you need is the constants for colors. One
example is this file which contains only the constants, nothing else.
http://github.com/rkumar/todorb/blob/master/lib/co...

once you've included it, all you need to do to print red is:

text = "hello there"

puts "#{RED}#{text}#{CLEAR}"

There are constants for colors, attributes, backgrounds etc.
0026dd77fd9ecc97b36e5b79cdbcf590?d=identicon&s=25 R. Kumar (sentinel)
on 2010-06-18 12:08
Brian Candler wrote:
> Rein Henrichs wrote:
>> There are probably another dozen tools I haven't found yet or forgot
>> didn't mention.
>
> There's one I can never remember the name of, but is something to do
> with prostitutes. Oh yes, it's "trollop".

William Morgan, hehe :-)

I just did search on github and found this :

http://awesome-cli-ruby.heroku.com/ by Dave Copeland.

Looks pretty good. I am about to give it a spin.
0026dd77fd9ecc97b36e5b79cdbcf590?d=identicon&s=25 R. Kumar (sentinel)
on 2010-06-18 12:55
> I just did search on github and found this :
>
> http://awesome-cli-ruby.heroku.com/ by Dave Copeland.
>
> Looks pretty good. I am about to give it a spin.

Okay, I just ran it a bit. Looks good. You might consider trying the
presentation (link above) and seeing if it helps you.


From the docs http://davetron5000.github.com/gli/:

What this doesn’t give you:

    * A way to indicate required flags
    * A way to indicate a required argument or required number of
arguments
    * A way to do default switches to ‘true’ and therefore accept things
like --no-force
    * A way to have repeated flags turn into an array or other
type-transforming things

What this gives you:

    * A reasonably useful help system. your_program help will list all
the global options and commands (along with command aliases) and
your_program help command_name will list help for that given command.
    * Error handling when flags do not receive arguments or unknown
flags or switches are given
    * Error handling when an unknown command is specified
    * Default values for flags if they are not specified by the user
(switches all default to false)
    * An easy way to allow location-specific defaults for options via a
config file for your app
0026dd77fd9ecc97b36e5b79cdbcf590?d=identicon&s=25 R. Kumar (sentinel)
on 2010-06-18 18:51
John Feminella wrote:

> Does that sort of thing exist? I've noticed there are some apparently
> more powerful alternatives like Thor, which is more of a full-stack
> scripting framework, and I'm wondering what else might be useful to
> take a look at.
>
> Any and all suggestions would be much appreciated. Thanks!

John,
I've done a quick comparison of Commander and GLI (git like interface)
(see prev post - "awesome cli").

1. Commander more downloads 3748, gli 1629 (source gemcutter).

2. gli offers scaffolding (generation of dir structure, and a program to
start off with containing tasks specified on command line.)

   commander does not (from what i see), but I've written a similar ruby
script to generate the shell program.

3. Commander and gli both offer git like commands and help etc. However,
i think gli's command line parsing is not as complete as Commander,
which wraps over OptionParser - so if you know Optionparser you can
easily use it, and you don't lose what you are used to.

Both offer command aliasing, but it seems commander is better. Commander
allows you to alias a shortcut to a complex command.

4. Commander wraps over Highline, but i don't see that as a huge issue.
You can always use highline if you want separately.
 Commander also has a decent Progressbar and a terminal-table output
formatter.

5. GLI can read your program and generate an rdoc which gives help and
options. I don't see that in commander. Not v important, though.

6. GLI generates a directory structure, with gemspec/Rakefile etc,
Commander does not. I personally am using jeweler to create a project,
so I would rather not have to resolve between these 2.

7. GLI has pre and post handlers and on_error: if needed that can be
added in one's script anyway, i think.

All in all, after this quick comparison, it seems commander has an edge
for *my* needs. Yours could differ.

I'll probably start porting my next shell app to ruby in a few days
using Commander. HTH.
0026dd77fd9ecc97b36e5b79cdbcf590?d=identicon&s=25 R. Kumar (sentinel)
on 2010-06-18 18:59
> 2. gli offers scaffolding (generation of dir structure, and a program to
> start off with containing tasks specified on command line.)
>
>    commander does not (from what i see), but I've written a similar ruby
> script to generate the shell program.
>

Commander does offer generating a basic program:

     $ commander init
0026dd77fd9ecc97b36e5b79cdbcf590?d=identicon&s=25 R. Kumar (sentinel)
on 2010-06-20 12:47
Attachment: opt.rb (1 KB)
Thomas Sawyer wrote:

>
> OptionParser generally works fine --even for the items you mention. To
> handle subcommands just pop off the top of ARGV before parsing (or for
> more advanced use, find the first non-option item).

I just found a great example of how you can use subcommands with
OptionParser.

http://stackoverflow.com/questions/2732894/using-r...

Note that neither this nor trollop will give you information of
subcommands on just typing help. We will have to manually add the help
text, but that's a small price to pay for the simplicity of either. I've
attached a tiny sample. You can run it as :
ruby opt.rb --help
ruby opt.rb foo --help
ruby opt.rb foo -q
E0d864d9677f3c1482a20152b7cac0e2?d=identicon&s=25 Robert Klemme (Guest)
on 2010-06-20 19:51
(Received via mailing list)
On 20.06.2010 12:47, R.. Kumar wrote:
> Thomas Sawyer wrote:
>
>> OptionParser generally works fine --even for the items you mention. To
>> handle subcommands just pop off the top of ARGV before parsing (or for
>> more advanced use, find the first non-option item).
>
> I just found a great example of how you can use subcommands with
> OptionParser.
>
> 
http://stackoverflow.com/questions/2732894/using-r...

I would modify that approach because it has the drawback of wasting CPU
cycles creating all the unnecessary OptionParsers for subcommands that
are not found in ARGV.  Rather, I'd do something more lazy, e.g.

global = OptionParser.new do |opts|
   # ...
end

subcommands = {
   'foo' => lambda {|argv| OptionParser.new do |opts|
      # ...
    end.parse! argv},
    # ...
    'baz' => lambda {|argv| OptionParser.new do |opts|
      # ...
    end.parse! argv},
  }

  global.order!
  subcommands[ARGV.shift][ARGV]

Of course you can also make "subcommands" a method which figures what to
return based on the argument (command name).

Kind regards

  robert
0026dd77fd9ecc97b36e5b79cdbcf590?d=identicon&s=25 R. Kumar (sentinel)
on 2010-06-20 20:25
Robert Klemme wrote:

>
> I would modify that approach because it has the drawback of wasting CPU
> cycles creating all the unnecessary OptionParsers for subcommands that
> are not found in ARGV.  Rather, I'd do something more lazy, e.g.
>
> global = OptionParser.new do |opts|
>    # ...
> end
>
> subcommands = {
>    'foo' => lambda {|argv| OptionParser.new do |opts|
>       # ...
>     end.parse! argv},
>     # ...
>     'baz' => lambda {|argv| OptionParser.new do |opts|
>       # ...
>     end.parse! argv},
>   }
>
>   global.order!
>   subcommands[ARGV.shift][ARGV]
>
> Of course you can also make "subcommands" a method which figures what to
> return based on the argument (command name).
>
> Kind regards
>
>   robert

I have actually isolated this into a seperate module with a method named
"command" which creates the subcommand. It also takes care of :

   prog help command

which trollop does not to my knowledge. It also prints out the
subcommands with their descriptions when typing help or --help just as
git does (which trollop does not.)

http://gist.github.com/445992

I will look into your feedback about lazy creation and update the same.
Could you have a quick look at the gist attached since my ruby is quite
newbie level.
0026dd77fd9ecc97b36e5b79cdbcf590?d=identicon&s=25 R. Kumar (sentinel)
on 2010-06-20 21:02
R.. Kumar wrote:
> Robert Klemme wrote:
>
>>
>> I would modify that approach because it has the drawback of wasting CPU
>> cycles creating all the unnecessary OptionParsers for subcommands that
>> are not found in ARGV.  Rather, I'd do something more lazy, e.g.
>>

Robert,
I have updated my gist to make it lazy

http://gist.github.com/446014

However, there is one issue. When typing help on the main program such
as

    $ ruby prog help
...
    Commands are:

    foo : does foo
    baz : does baz

I was being shown the subcommands and their descriptions, since I had
parsed all the optionparsers. Now that I only parse if a particular
command is called, i cannot get the description of the command. (search
XXX in source).

Is there any way of doing this ?

Regards
RK
E0d864d9677f3c1482a20152b7cac0e2?d=identicon&s=25 Robert Klemme (Guest)
on 2010-06-21 13:08
(Received via mailing list)
2010/6/20 R.. Kumar <sentinel1879@gmail.com>:
> I have updated my gist to make it lazy
>    foo : does foo
>    baz : does baz
>
> I was being shown the subcommands and their descriptions, since I had
> parsed all the optionparsers. Now that I only parse if a particular
> command is called, i cannot get the description of the command. (search
> XXX in source).
>
> Is there any way of doing this ?

Well, either you revert back to creating all sub OptionParsers or you
store the description in some other place.

Kind regards

robert
0026dd77fd9ecc97b36e5b79cdbcf590?d=identicon&s=25 R. Kumar (sentinel)
on 2010-06-21 13:33
Robert Klemme wrote:

>>
>> Is there any way of doing this ?
>
> Well, either you revert back to creating all sub OptionParsers or you
> store the description in some other place.
>
> Kind regards
>
> robert

Actually i did find a way out !!

Earlier in the loop, I was doing:
             desc = opt.description

Now I do: opt.call.description

      cmdtext = "Commands are:"
      @commands.each_pair do |c, opt|
        desc = opt.call.description
        cmdtext << "\n   #{c} : #{desc}"
      end

Works.

I modified my app to use this, it was easy since both use OptionParser.
I added aliases. But my app has slightly more complex aliases where one
command maps to multiple words. Working on that currently.
8f0660cdc9f5d91c7d97456f8f0be8c7?d=identicon&s=25 Gabriel Horner (cldwalker)
on 2010-06-27 03:07
I keep lists of related gems on delicious that could help.

> * colorized output

http://delicious.com/tag/gem:cs=color

> * sensible option parsing

http://delicious.com/tag/gem:type=opt


> Does that sort of thing exist? I've noticed there are some apparently
> more powerful alternatives like Thor, which is more of a full-stack
> scripting framework, and I'm wondering what else might be useful to
> take a look at.

More libraries like thor at http://delicious.com/tag/gem:type=task

I'd vote for thor over commander for most of what you need. Much more
widely used and less verbose DSL.

Gabriel
0026dd77fd9ecc97b36e5b79cdbcf590?d=identicon&s=25 R. Kumar (sentinel)
on 2010-06-27 11:24
Gabriel Horner wrote:

>
> I'd vote for thor over commander for most of what you need. Much more
> widely used and less verbose DSL.
>
> Gabriel

I've been looking into more detail into the thor documentation and the
specs (http://rdoc.info/projects/wycats/thor).

1. Does it cover what Highline provides ? I notice some methods like
ask? etc, but does it create menus, get password, dates etc.

2. For option parsing does it use OptionParser? (deos not seem to). Or
like most other alternatives does it provide a subset of optionparser ?

3. What exactly does Shell mean in thor context.
(http://rdoc.info/projects/wycats/thor). I tried running the example on
this page but get an error:

/opt/local/lib/ruby1.9/gems/1.9.1/gems/thor-0.13.6/lib/thor/runner.rb:34:in
`method_missing': undefined method `start' for nil:NilClass
(NoMethodError)

That's perhaps since there's no task in it, just a module. So i have no
idea what it does.
This topic is locked and can not be replied to.