[RFC] Proposing a Ruby Packaging Standard

Ruby
1

Hello,

earlier this month I wrote up a draft for a specification on how to
structure Ruby projects. It started at RPS-SPEC · GitHub
and there was lots of discussion there, which I now want to move to
ruby-talk to gain a larger audience.

The specification now resides at

             http://chneukirchen.github.com/rps/

and is generated from GitHub - leahneukirchen/rps: Ruby Packaging Standard.

Please keep discussion on RPS to this thread, so people which usually
don’t follow ruby-talk can discuss too.

Thank you,

2

On Apr 20, 3:35 am, Christian N. [email protected]
wrote:

and is generated fromhttp://github.com/chneukirchen/rps.

Please keep discussion on RPS to this thread, so people which usually
don’t follow ruby-talk can discuss too.

I don’t understand. Do you want us to discuss it here or not?

3

On Apr 20, 11:00 am, Intransition [email protected] wrote:

I don’t understand. Do you want us to discuss it here or not?

Ok. Just read last post from github and it indicates to discuss here.
Also, I noticed the the Google archive knocked off the [RFC] portion
of the subject. Hope that’s not problem.

4

Final comments for now, I promise :wink:

  1. Change “Executables SHOULD NOT require ‘rubygems’ or modify the
    $LOAD_PATH” to a “MUST NOT”. There’s no reason to do this in an
    executable.

  2. Change “Libraries SHOULD NOT require code of the project that are
    outside of lib/” and “Executables SHOULD NOT require code of the
    project that are outside of lib/” to a “MUST NOT”. Otherwise other
    package mangers like setup.rb and rip will not work.

  3. I have basically given up on using data/ directory. I now put all
    such files in lib/. It is much easier to use relative lookup then to
    use rbconfig.rb and Config::CONFIG[‘datadir’]. While some think this
    is a “no no”, I don’t see any worthy advantage of using data/. I know
    the noble idea is that data/ is then accessible by other libraries,
    but honestly it almost never happens. In a case where it is likely,
    then ok use data/, but I don’t think it should be mandatory. So I
    suggest removing “Ruby library files MUST end with .rb.” altogether.

  4. Test locations might be a little too framework-oriented for any
    kind of RPS. Test::Unit and MiniTest use test/, RSpec used spec/,
    Cucumber uses features/. I ran into this question when developing QED
    and in the end I just settled for qed/.

5

In response josh and zimbatm regarding setup.rb.

I carried on development of that project here:
http://proutils.github.com/setup.
It can now run as ordinary bin, or, of course, it can be included as a
project script when specifically necessary.

Point responses to zimbatm’s post

  • data/ is copied in #{PREFIX_DIR}/share (like /usr/share). Why not name it “share” then ?

I would agree, but rbconfig.rb refers to it as ‘datadir’.

  • conf/ is copied in /etc. Why not name it “etc” then ?

The latest versions of setup.rb have renamed that to etc/.

  • setup.rb multi-project layout is probably a bad idea. Package management should handle this

Latest versions of setup.rb no longer support multi-project layout.

The most problematic issue I had with working on setup.rb was with
documentation. FHS standards would indicate that the packages
documentation be copied to /usr/share/doc/{name}/ (using standard
system prefix), However, unless Ruby projects use doc/{name}/ (just
like lib/{name}/) then it’s not very feasible b/c there is no standard
way to get the name of the project. Right now it looks in
either .setup/name or .meta/name (or without dot) as a stop-gap
measure for lack of any other option. If anything comes out this
process, beyond just writing down current common practices, maybe it
would be this doc/{name}/ convention.

On the other hand, maybe it’s time to give setup.rb and the whole FHS
style of program installation the kiss of death.

6

Hi,

I’ve only ever used RubyGems for packaging my projects, so I’m kind of
scared of setup.rb’s style of separating my project’s directories (i.e.
lib/ is no longer next to bin/ and so on). In particular, I am confused
about the following:

I want to run the man executable (pointing it to my project’s man/
directory) from my project’s executable in bin/. With RubyGems, I can
rely on bin/ being next to man/. But with setup.rb I don’t know how to
find my project’s man/ directory from my project’s executable in bin/!

Any suggestions?

Also, if I provide a man/ directory in my gem/tarball package like this:

man/
└── man1
├── ember.1
└── ember.1.html

1 directory, 2 files

Will it be copied into /usr/share/man/ by RubyGems/setup.rb/Rip and
friends?

Thanks for your consideration.

7

Intransition wrote:

  1. Change “Executables SHOULD NOT require ‘rubygems’ or modify the
    $LOAD_PATH” to a “MUST NOT”. There’s no reason to do this in an
    executable.

Except for the gem executable itself, or any other executable that
needs to operate on gems. SHOULD NOT was the right call.

8

On Apr 20, 12:35 pm, Joel VanderWerf [email protected] wrote:

Intransition wrote:

  1. Change “Executables SHOULD NOT require ‘rubygems’ or modify the
    $LOAD_PATH” to a “MUST NOT”. There’s no reason to do this in an
    executable.

Except for the gem executable itself, or any other executable that
needs to operate on gems. SHOULD NOT was the right call.

Clearly the gem command itself is necessary exception --it is
rubygems. I would suggest any other library that “operates on gems”
still use a lib/ file and require that in their executable. But the
rule applies even in lib/, so here again it is an exceptional case
when one is specifically extending/augmenting rubygems. The rules
intent is to address all other cases. So perhaps the best phrasing is
still “MUST” but with a qualification concerning libraries designed
for extending/augmenting rubygems.

9

Intransition wrote:

Final comments for now, I promise :wink:

  1. Change “Executables SHOULD NOT require ‘rubygems’ or modify the
    $LOAD_PATH” to a “MUST NOT”. There’s no reason to do this in an
    executable.

I have a JRuby desktop app that clears out and then sets up $: The
reason is that I need to know that it is not loading anything other than
what is bundled up with the application.

If there is a way to restrict the load path without altering $: I’d be
interested to know what it is.

–
James B.

www.jamesbritt.com - Playing with Better Toys
www.ruby-doc.org - Ruby Help & Documentation
www.rubystuff.com - The Ruby Store for Ruby Stuff
www.neurogami.com - Smart application development

10

On Apr 20, 4:27 am, Christian N. [email protected]
wrote:

and is generated fromhttp://github.com/chneukirchen/rps.

Please keep discussion on RPS to this thread, so people which usually
don’t follow ruby-talk can discuss too.

My comments:

“Extensions” Section:

  1. “Extensions SHOULD reside in ext/”

There are project that include more than one extension, I’ve
encouraged project follow the setup.rb folder structure, exemplified
here:

(under structure)

  1. “Extensions SHOULD be buildable with ruby extconf.rb; make.”

That is not true, there are extension that even using extconf they
generate Rakefiles, because they depend on mkrf and not mkmf

That can’t be enforced on these developers, if done a lot of project
could break.

  1. “Extensions SHOULD be installed into an architecture-specific
    directory.”

RubyGems doesn’t do that, it just relocate binaries from ext directory
to lib, adding the platform directory will make impossible for pre-
built binary gems that bundle 1.8 and 1.9 extension.

It also means that RubyGems will need to add RUBY_PLATFORM (or
RbConfig::CONFIG[‘host_os’]) of each gem into $LOAD_PATH.

This has been requested in the past in RubyGems:

http://rubyforge.org/tracker/index.php?func=detail&aid=14943&group_id=126&atid=575

Installations not using RubyGems will need to adapt their libraries to
try loading from RUBY_PLATFORM directory and under failure try from
other options?

If no RubyGems and no developers implementing those tricks, means Ruby
will need to be modified to handle that.

Setup.rb already handles that putting binaries outside the package
and inside site_ruby for each RUBY_PLATFORM

  1. Files ending with .exe

AFAIK: "Exe"cutables, not extensions or libraries

My two cents

11

On Apr 20, 1:12Â pm, Suraj K. [email protected] wrote:

find my project’s man/ directory from my project’s executable in bin/!

Any suggestions?

The standard approach would be something like:

require ‘rbconfig’
dir = Config::CONFIG[‘datadir’]
file = File.join(dir, ‘man/man1/ember.1’)

B/c of RubyGems there is an extension method that redirects to the the
appropriate place.

Config.datadir(“ember”)

But I think it might be a problem to tell which is which for locating
the man/ directory.

Also, if I provide a man/ directory in my gem/tarball package like this:

man/
└── man1
  ├── ember.1
  └── ember.1.html

Can man handle .html? I glanced at the entries in my system and I
noticed they are all gzipped (ending in .gz).

Will it be copied into /usr/share/man/ by RubyGems/setup.rb/Rip and
friends?

Setup.rb does. RubyGems does not. Not sure about Rip.

12

Thomas S. wrote:

On Apr 20, 1:12Â pm, Suraj K. [email protected] wrote:

find my project’s man/ directory from my project’s executable in bin/!

The standard approach would be something like:

require ‘rbconfig’
dir = Config::CONFIG[‘datadir’]
file = File.join(dir, ‘man/man1/ember.1’)

Thanks. Based on this example, I found that there is a
Config::CONFIG[‘mandir’] which tells us exactly where the
system’s man/ directory is located.

B/c of RubyGems there is an extension method that redirects to the the
appropriate place.

Config.datadir(“ember”)

RubyGems does not seem to provide that functionality on my system:

sun@yantram ~> irb
## ruby 1.9.1p378 (2010-01-10 revision 26273) [i686-linux]
>> require 'rbconfig'
=> true
>> Config.datadir("ember")
NoMethodError: undefined method `datadir' for RbConfig:Module
        from (irb):2
        from /usr/bin/irb:12:in `<main>'
>> require 'rubygems'
=> false
>> Config.datadir("ember")
NoMethodError: undefined method `datadir' for RbConfig:Module
        from (irb):4
        from /usr/bin/irb:12:in `<main>'
>> Gem::VERSION
=> "1.3.6"

But I think it might be a problem to tell which is which for locating
the man/ directory.

I guess RubyGems provides a Config.mandir(the_gem_name) method to do
this?

Also, if I provide a man/ directory in my gem/tarball package like this:

man/
└── man1
  ├── ember.1
  └── ember.1.html

Can man handle .html?

Nope. That’s just there as a backup, in case a user is on Windoze
or their system somehow lacks the man command. man does warn me
about it though when searching for manpages in my man/ directory:

man: warning: man/man1/ember.1.html: ignoring bogus filename

But I think it’s worthy trade-off to have a HTML version available.

I glanced at the entries in my system and I
noticed they are all gzipped (ending in .gz).

Thanks for the tip. I’ve gzipped my Roff manpage now.

Will it be copied into /usr/share/man/ by RubyGems/setup.rb/Rip and
friends?

Setup.rb does. RubyGems does not. Not sure about Rip.

Will RubyGems ever follow the FHS style of installation like setup.rb?

Will setup.rb ever follow the keep-everything-together RubyGems style?

What installation style will this Ruby Packaging Standard follow?

So many questions… Life was easier when I only used RubyGems. :-/

13

ghorner wrote:

I want to run the man executable (pointing it to my project’s man/

Just use gem-man, GitHub - defunkt/gem-man: RubyGems plugin to view a gem's manpage.

Excellent tip! Thank you.

Will it be copied into /usr/share/man/ by RubyGems/setup.rb/Rip

The next version of rip should handle installing man pages.

This is great news. I hope it also gzips man pages (if they’re not
already) during package installation to save space.

14

I want to run the man executable (pointing it to my project’s man/
directory) from my project’s executable in bin/. With RubyGems, I can
rely on bin/ being next to man/. But with setup.rb I don’t know how to
find my project’s man/ directory from my project’s executable in bin/!

Any suggestions?

Just use gem-man, http://github.com/defunkt/gem-man:

$ gem install gem-man
$ gem man ember

Will it be copied into /usr/share/man/ by RubyGems/setup.rb/Rip and
friends?

The next version of rip should handle installing man pages.

15

Lucas N. wrote:

On 21/04/10 at 14:01 +0900, Suraj K. wrote:

Will RubyGems ever follow the FHS style of installation like setup.rb?
Will setup.rb ever follow the keep-everything-together RubyGems style?
What installation style will this Ruby Packaging Standard follow?

I think that what is important is that we have a clear set of
guidelines for Ruby library developers to follow, that allow to do
both rubygems-style stuff and setup.rb-style stuff.

It’s burdensome on Ruby library developers to support both ways.

Perhaps this Ruby Packaging Standard could provide a tiny Ruby
library to ease that burden, by making the FHS/setup.rb style
installation appear like RubyGems style installation (i.e. how all
files are organized inside a release package and when checked out
from a source code repository)?

The goal of this “standard” should not be to push one or the other
solution.

Good point. Thanks for clarifying this.

16

On 21/04/10 at 14:01 +0900, Suraj K. wrote:

Thanks. Based on this example, I found that there is a
sun@yantram ~> irb
NoMethodError: undefined method `datadir’ for RbConfig:Module

or their system somehow lacks the man command. man does warn me

So many questions… Life was easier when I only used RubyGems. :-/

I think that what is important is that we have a clear set of guidelines
for Ruby library developers to follow, that allow to do both
rubygems-style stuff and setup.rb-style stuff. The goal of this
“standard” should not be to push one or the other solution.

17

On 21/04/10 at 14:55 +0900, Suraj K. wrote:

It’s burdensome on Ruby library developers to support both ways.
Not really. Both solutions are largely compatible. You just need to
organize the files correctly, and it just works.

Perhaps this Ruby Packaging Standard could provide a tiny Ruby
library to ease that burden, by making the FHS/setup.rb style
installation appear like RubyGems style installation (i.e. how all
files are organized inside a release package and when checked out
from a source code repository)?

I’m not sure I understand your point. The point of setup.rb is to
respect FHS, which contradicts installing everything in the same place.
But the starting point of a setup.rb installation is the same as a
well-organized gem: if you put executables in bin/, libraries in lib/,
etc, it just works.

18

Lucas N. wrote:

On 21/04/10 at 14:55 +0900, Suraj K. wrote:

Perhaps this Ruby Packaging Standard could provide a tiny Ruby
library […] making the FHS/setup.rb style installation appear
like RubyGems style installation

I’m not sure I understand your point. The point of setup.rb is to
respect FHS, which contradicts installing everything in the same place.

My concern is about the Ruby code in bin/ and lib/ being able to
find the rest of the files that originally came in the package. If
this RPS would define that related-file-finding code, then I don’t
need to reinvent it (or copy/paste) for all of my Ruby libraries.

19

Luis L. [email protected] writes:

  1. Files ending with .exe

AFAIK: "Exe"cutables, not extensions or libraries

Afaict do libraries have that extension on VMS.

20

Dmitry B. wrote:

On Wed, Apr 21, 2010 at 9:36 AM, Suraj K. [email protected]
wrote:

My concern is about the Ruby code in bin/ and lib/ being able to
find the rest of the files that originally came in the package.

Isn’t that precisely what Config::CONFIG[‘datadir’] is for?

Yes, but that value does not reflect RubyGems style on my system:

Config::CONFIG[‘datadir’]
=> “/usr/share”

gem env.lines.grep(/install/i)
=> [" - INSTALLATION DIRECTORY: /usr/lib/ruby/gems/1.9.1\n"]

It needs to reflect both styles to work correctly under either one.

I am using RubyGems 1.3.6 with Ruby 1.9.1p378 on i686-linux.