State of the union for Ruby CLI libraries?

[johnf]

I am starting construction on a somewhat complicated internal
application whose primary interface will be the command line. In the
past, for similar projects, I’ve just used OptionParser, but I think I
need something a little beefier. Specifically, I think I need these
features:

• colorized output
• sensible option parsing
• sensible enumeration of options for help output
• support for git-like subcommands, e.g., app create, app delete, etc., that can be intuitively and cleanly mapped to methods
• ability to be insensitive to option ordering, e.g. app create --foo quux --bar should be semantically identical to app create --bar --foo quux

Does that sort of thing exist? I’ve noticed there are some apparently
more powerful alternatives like Thor, which is more of a full-stack
scripting framework, and I’m wondering what else might be useful to
take a look at.

Any and all suggestions would be much appreciated. Thanks!

[johnf]

On Jun 13, 12:07 pm, John F. [email protected] wrote:

delete`, etc., that can be intuitively and cleanly mapped to methods

• ability to be insensitive to option ordering, e.g. app create --foo quux --bar should be semantically identical to app create --bar --foo quux

Does that sort of thing exist? I’ve noticed there are some apparently
more powerful alternatives like Thor, which is more of a full-stack
scripting framework, and I’m wondering what else might be useful to
take a look at.

Any and all suggestions would be much appreciated. Thanks!

OptionParser generally works fine --even for the items you mention. To
handle subcommands just pop off the top of ARGV before parsing (or for
more advanced use, find the first non-option item). The order of
options is not an issue, and help is pretty easy since OptionPaser
responds to #to_s. Running that through a colorizing filter shouldn’t
be too hard. For color try http://rubyworks.github.com/ansi.

There are other options out there though, check out HighLine and
Oyster for instance.

[johnf]

Rein H. wrote:

There are probably another dozen tools I haven’t found yet or forgot
didn’t mention.

There’s one I can never remember the name of, but is something to do
with prostitutes. Oh yes, it’s “trollop”.

Examples at http://trollop.rubyforge.org/ suggest that subcommands are
supported.

[johnf]

On 2010-06-13 09:07:31 -0700, John F. said:

delete`, etc., that can be intuitively and cleanly mapped to methods

• ability to be insensitive to option ordering, e.g. app create --foo quux --bar should be semantically identical to app create --bar --foo quux

Does that sort of thing exist? I’ve noticed there are some apparently
more powerful alternatives like Thor, which is more of a full-stack
scripting framework, and I’m wondering what else might be useful to
take a look at.

Any and all suggestions would be much appreciated. Thanks!

After recently reviewing the state of the art for Ruby CLI tools myself
as we moved Puppet to a sub-command-based executable structure, I
couldn’t find a single, cohesive library that provides all of these
features. The domain is a full of NIH and wheel reinvention. There also
doesn’t seem to be much thought given to combining multiple tools in a
unix-like fashion (small tools chained together to provide more
complete functionality). They all seem to want to do a bit of
everything but not enough of anything.

OptionParser is convenient because it’s available in the standard
library. There are a number of other option parsing tools but they
(imo) don’t provide significant benefit to offset the cost of an
additional dependency.

As far as subcommands go, there is rake (which can be used as a
stand-alone application DSL), thor (by Yehuda K.), main (by Ara
Howard), the Command code that is a part of the github gem (by Chris
Wanstrath), the Application code that is a part of Puppet and I’m sure
at least a dozen other reimplementations hidden in various gems and
libraries.

I found Rake, Thor and Main gave me too much friction for the type of
CLI tool I was interested in. Option parsing DSLs that were cumbersome,
cryptic error handling and unweildy API design for my use case. That
said, please try them and see if they work for you.

There are probably another dozen tools I haven’t found yet or forgot
didn’t mention. Github queries for “Ruby CLI” and similar return a
staggering number of results. I’d welcome any suggestions for my own
use as well.

Finally I recently wrote a very small sub-command DSL gem[1] in a fit
of my own NIH. It almost certainly doesn’t do what you wantbut you are
welcome to try it. The design goal was to do one thing (subcommands)
and do it well, and to get out of your way and let you use your own
tools for everything else. I would welcome feedback.

[1] GitHub - reinh/commandant: A simple library for writing commands with subcommands (like git)

[johnf]

John F. wrote:

• colorized output

That’s a trivial issue. All you need is the constants for colors. One
example is this file which contains only the constants, nothing else.
http://github.com/rkumar/todorb/blob/master/lib/common/colorconstants.rb

once you’ve included it, all you need to do to print red is:

text = “hello there”

puts “#{RED}#{text}#{CLEAR}”

There are constants for colors, attributes, backgrounds etc.

[johnf]

John F. wrote:

Does that sort of thing exist? I’ve noticed there are some apparently
more powerful alternatives like Thor, which is more of a full-stack
scripting framework, and I’m wondering what else might be useful to
take a look at.

Any and all suggestions would be much appreciated. Thanks!

There’s also visionmedia’s commander
GitHub - tj/commander: The complete solution for Ruby command-line executables.

I’ve played with it a bit, but not started working on it. I’ve started
converting some shell-scripts to ruby, and i’ve rolled my first one
using optionparser.

I may try commander for the second one. But I am afraid of a large code
base, and it getting broken at some stage. For the ease of some option
parsing and a few more tiny goodies, I can;t have my entire app
dependent on some large code base.

My current apps manipulate files largely using “sed”, “grep”, “cut” and
some other unix commands. So the meat of my programs is still not served
by these frameworks.

I once tried Thor and liked it. But its written by Y Katz who is into
Rails currently. So i don’t know if it’s being maintained (if you run
into issues). I;ve been bitten earlier using a gem written by a high
profile programmer who was too involved elsewhere to fix a bug in his
gem or even reply.

[johnf]

Brian C. wrote:

Rein H. wrote:

There are probably another dozen tools I haven’t found yet or forgot
didn’t mention.

There’s one I can never remember the name of, but is something to do
with prostitutes. Oh yes, it’s “trollop”.

William M., hehe

I just did search on github and found this :

http://awesome-cli-ruby.heroku.com/ by Dave Copeland.

Looks pretty good. I am about to give it a spin.

[johnf]

John F. wrote:

Does that sort of thing exist? I’ve noticed there are some apparently
more powerful alternatives like Thor, which is more of a full-stack
scripting framework, and I’m wondering what else might be useful to
take a look at.

Any and all suggestions would be much appreciated. Thanks!

John,
I’ve done a quick comparison of Commander and GLI (git like interface)
(see prev post - “awesome cli”).

1. Commander more downloads 3748, gli 1629 (source gemcutter).

2. gli offers scaffolding (generation of dir structure, and a program to
   start off with containing tasks specified on command line.)

   commander does not (from what i see), but I’ve written a similar ruby
   script to generate the shell program.

3. Commander and gli both offer git like commands and help etc. However,
   i think gli’s command line parsing is not as complete as Commander,
   which wraps over OptionParser - so if you know Optionparser you can
   easily use it, and you don’t lose what you are used to.

Both offer command aliasing, but it seems commander is better. Commander
allows you to alias a shortcut to a complex command.

4. Commander wraps over Highline, but i don’t see that as a huge issue.
   You can always use highline if you want separately.
   Commander also has a decent Progressbar and a terminal-table output
   formatter.

5. GLI can read your program and generate an rdoc which gives help and
   options. I don’t see that in commander. Not v important, though.

6. GLI generates a directory structure, with gemspec/Rakefile etc,
   Commander does not. I personally am using jeweler to create a project,
   so I would rather not have to resolve between these 2.

7. GLI has pre and post handlers and on_error: if needed that can be
   added in one’s script anyway, i think.

All in all, after this quick comparison, it seems commander has an edge
for my needs. Yours could differ.

I’ll probably start porting my next shell app to ruby in a few days
using Commander. HTH.

[johnf]

I just did search on github and found this :

http://awesome-cli-ruby.heroku.com/ by Dave Copeland.

Looks pretty good. I am about to give it a spin.

Okay, I just ran it a bit. Looks good. You might consider trying the
presentation (link above) and seeing if it helps you.

From the docs http://davetron5000.github.com/gli/:

What this doesn’t give you:

* A way to indicate required flags
* A way to indicate a required argument or required number of 

arguments
* A way to do default switches to ‘true’ and therefore accept things
like --no-force
* A way to have repeated flags turn into an array or other
type-transforming things

What this gives you:

* A reasonably useful help system. your_program help will list all 

the global options and commands (along with command aliases) and
your_program help command_name will list help for that given command.
* Error handling when flags do not receive arguments or unknown
flags or switches are given
* Error handling when an unknown command is specified
* Default values for flags if they are not specified by the user
(switches all default to false)
* An easy way to allow location-specific defaults for options via a
config file for your app

[johnf]

Thomas S. wrote:

OptionParser generally works fine --even for the items you mention. To
handle subcommands just pop off the top of ARGV before parsing (or for
more advanced use, find the first non-option item).

I just found a great example of how you can use subcommands with
OptionParser.

Using ruby's OptionParser to parse sub-commands

Note that neither this nor trollop will give you information of
subcommands on just typing help. We will have to manually add the help
text, but that’s a small price to pay for the simplicity of either. I’ve
attached a tiny sample. You can run it as :
ruby opt.rb --help
ruby opt.rb foo --help
ruby opt.rb foo -q

[johnf]

2. gli offers scaffolding (generation of dir structure, and a program to
   start off with containing tasks specified on command line.)

   commander does not (from what i see), but I’ve written a similar ruby
   script to generate the shell program.

Commander does offer generating a basic program:

 $ commander init

[johnf]

On 20.06.2010 12:47, R… Kumar wrote:

Thomas S. wrote:

OptionParser generally works fine --even for the items you mention. To
handle subcommands just pop off the top of ARGV before parsing (or for
more advanced use, find the first non-option item).

I just found a great example of how you can use subcommands with
OptionParser.

optparse - Using ruby's OptionParser to parse sub-commands - Stack Overflow

I would modify that approach because it has the drawback of wasting CPU
cycles creating all the unnecessary OptionParsers for subcommands that
are not found in ARGV. Rather, I’d do something more lazy, e.g.

global = OptionParser.new do |opts|

…

end

subcommands = {
‘foo’ => lambda {|argv| OptionParser.new do |opts|
# …
end.parse! argv},
# …
‘baz’ => lambda {|argv| OptionParser.new do |opts|
# …
end.parse! argv},
}

global.order!
subcommands[ARGV.shift][ARGV]

Of course you can also make “subcommands” a method which figures what to
return based on the argument (command name).

Kind regards

robert

[johnf]

Robert K. wrote:

I would modify that approach because it has the drawback of wasting CPU
cycles creating all the unnecessary OptionParsers for subcommands that
are not found in ARGV. Rather, I’d do something more lazy, e.g.

global = OptionParser.new do |opts|

…

end

subcommands = {
‘foo’ => lambda {|argv| OptionParser.new do |opts|
# …
end.parse! argv},
# …
‘baz’ => lambda {|argv| OptionParser.new do |opts|
# …
end.parse! argv},
}

global.order!
subcommands[ARGV.shift][ARGV]

Of course you can also make “subcommands” a method which figures what to
return based on the argument (command name).

Kind regards

robert

I have actually isolated this into a seperate module with a method named
“command” which creates the subcommand. It also takes care of :

prog help command

which trollop does not to my knowledge. It also prints out the
subcommands with their descriptions when typing help or --help just as
git does (which trollop does not.)

https://gist.github.com/rkumar/445992

I will look into your feedback about lazy creation and update the same.
Could you have a quick look at the gist attached since my ruby is quite
newbie level.

[johnf]

2010/6/20 R… Kumar [email protected]:

I have updated my gist to make it lazy
foo : does foo
baz : does baz

I was being shown the subcommands and their descriptions, since I had
parsed all the optionparsers. Now that I only parse if a particular
command is called, i cannot get the description of the command. (search
XXX in source).

Is there any way of doing this ?

Well, either you revert back to creating all sub OptionParsers or you
store the description in some other place.

Kind regards

robert

[johnf]

R… Kumar wrote:

Robert K. wrote:

I would modify that approach because it has the drawback of wasting CPU
cycles creating all the unnecessary OptionParsers for subcommands that
are not found in ARGV. Rather, I’d do something more lazy, e.g.

Robert,
I have updated my gist to make it lazy

https://gist.github.com/rkumar/446014

However, there is one issue. When typing help on the main program such
as

$ ruby prog help

…
Commands are:

foo : does foo
baz : does baz

I was being shown the subcommands and their descriptions, since I had
parsed all the optionparsers. Now that I only parse if a particular
command is called, i cannot get the description of the command. (search
XXX in source).

Is there any way of doing this ?

Regards
RK

[johnf]

Robert K. wrote:

Is there any way of doing this ?

Well, either you revert back to creating all sub OptionParsers or you
store the description in some other place.

Kind regards

robert

Actually i did find a way out !!

Earlier in the loop, I was doing:
desc = opt.description

Now I do: opt.call.description

  cmdtext = "Commands are:"
  @commands.each_pair do |c, opt|
    desc = opt.call.description
    cmdtext << "\n   #{c} : #{desc}"
  end

Works.

I modified my app to use this, it was easy since both use OptionParser.
I added aliases. But my app has slightly more complex aliases where one
command maps to multiple words. Working on that currently.

[johnf]

I keep lists of related gems on delicious that could help.

• colorized output

http://delicious.com/tag/gem:cs=color

• sensible option parsing

http://delicious.com/tag/gem:type=opt

Does that sort of thing exist? I’ve noticed there are some apparently
more powerful alternatives like Thor, which is more of a full-stack
scripting framework, and I’m wondering what else might be useful to
take a look at.

More libraries like thor at http://delicious.com/tag/gem:type=task

I’d vote for thor over commander for most of what you need. Much more
widely used and less verbose DSL.

Gabriel

[johnf]

Gabriel H. wrote:

I’d vote for thor over commander for most of what you need. Much more
widely used and less verbose DSL.

Gabriel

I’ve been looking into more detail into the thor documentation and the
specs (http://rdoc.info/projects/wycats/thor).

1. Does it cover what Highline provides ? I notice some methods like
   ask? etc, but does it create menus, get password, dates etc.

2. For option parsing does it use OptionParser? (deos not seem to). Or
   like most other alternatives does it provide a subset of optionparser ?

3. What exactly does Shell mean in thor context.
   (http://rdoc.info/projects/wycats/thor). I tried running the example on
   this page but get an error:

/opt/local/lib/ruby1.9/gems/1.9.1/gems/thor-0.13.6/lib/thor/runner.rb:34:in
method_missing': undefined method start’ for nil:NilClass
(NoMethodError)

That’s perhaps since there’s no task in it, just a module. So i have no
idea what it does.