Forum: Ruby pallet 1.2.1

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
9ff064364151151cbce108b869e6e62d?d=identicon&s=25 Stephen Touset (Guest)
on 2007-05-01 19:55
(Received via mailing list)
I've been writing a multiplatform packaging tool for the company I work
for, dubbed "pallet". I recently got the go-ahead to release it on
RubyForge under the LGPL, so have spent the past day or so getting it up
on there.

Essentially, pallet lets you use a simple Rakefile stanza (akin to the
existing gemspec stanza) to define multiple packaging formats. Right
now,
gem and dpkg are the only formats supported, although adding .tar.gz
should be trivial, and .rpm should be simple enough for anyone who's
built
an RPM.

An example of the format, from pallet's own Rakefile:

  Pallet.new('pallet', Pallet::VERSION) do |p|

    p.author  = 'Stephen Touset'
    p.email   = 'stouset@damballa.com'
    p.summary = 'Automated packaging tool for different languages and
OSes'

    p.packages << Pallet::Gem.new(p) do |gem|
      gem.depends = ['rake']
      gem.requirements = ['fakeroot', 'dpkg-dev']
      gem.files.include FileList['share/**/*']
    end

    p.packages << Pallet::Deb.new(p) do |deb|
      deb.architecture = 'all'
      deb.depends      = ['rake', 'fakeroot', 'dpkg-dev']
      deb.files        = [InstallSpec['lib',   '/usr/lib/ruby/1.8'],
                          InstallSpec['share', '/usr/share/pallet'],]
      deb.commands.document = 'rake doc'
    end

  end

It doesn't support every single possible feature of every packaging
system, but it's close enough for 99% of projects. Nearly thirty of our
own internal software projects were ported over to it as part of our
build
system in a matter of hours by two developers with no previous
experience
with either the packaging systems or pallet.

One nice ability is that it should require no more than maybe ten lines
to
build a Debian package that simply contains a gem that is
installed/uninstalled from rubygems by postinst/prerm scripts in the
Debian package. In a future release, I expect to make this a trivial
use-case, through something like

  p.packages << Pallet::Deb.new(p, :include => :gem)

It is extensively rdoc'd, available as "pallet" from rubygems, and can
be
downloaded as a dpkg or a gem from http://rubyforge.org/projects/pallet/

Happy packing!
9ff064364151151cbce108b869e6e62d?d=identicon&s=25 Stephen Touset (Guest)
on 2007-05-01 20:05
(Received via mailing list)
On Tue, 01 May 2007 12:53:18 -0500, Stephen Touset wrote:

> I've been writing a multiplatform packaging tool for the company I work
> for, dubbed "pallet". I recently got the go-ahead to release it on
> RubyForge under the LGPL, so have spent the past day or so getting it up
> on there.

As part of this release, I'd like to extend a (limited) offer to
personally package up a few -- maybe five to ten, time-willing --
willing
open source projects. Please reply to this message if you have a project
or
library that would benefit from being packaged as a .deb (preferably
also
a .gem). Higher-profile projects will be dealt with first.

Since I didn't include it in the original post, I want to clarify that
pallet can be used to package software from _any_ language and integrate
into any build system -- not just Ruby using Rake. Most of the projects
we
have at the company I developed this for are written in C built with
Makefiles or Java build with ant. None of these required any extra
effort by our developers to package.
Ffcb418e17cac2873d611c2b8d8d891c?d=identicon&s=25 unknown (Guest)
on 2007-05-02 10:40
(Received via mailing list)
Stephen:

> On Tue, 01 May 2007 12:53:18 -0500, Stephen Touset wrote:
>
>> I've been writing a multiplatform packaging tool for the company I
>> work
>> for, dubbed "pallet". I recently got the go-ahead to release it on
>> RubyForge under the LGPL, so have spent the past day or so getting it
>> up
>> on there.

*snip*

> Since I didn't include it in the original post, I want to clarify that
> pallet can be used to package software from _any_ language and integrate
> into any build system -- not just Ruby using Rake. Most of the projects
> we
> have at the company I developed this for are written in C built with
> Makefiles or Java build with ant. None of these required any extra
> effort by our developers to package.

This sounds extremely interesting Stephen: I would expect quite a lots
of interst, but there don't seem to be replies on the list. I wonder if
people haven't quite understood what you've released? Is it some kind of
meta package description that can be compiled in to vareious different
kinds of package? Or have I got completely the wrong end of the stick?

:-)

Cheers,
  Benjohn
753dcb78b3a3651127665da4bed3c782?d=identicon&s=25 Brian Candler (Guest)
on 2007-05-02 11:46
(Received via mailing list)
On Wed, May 02, 2007 at 02:55:07AM +0900, Stephen Touset wrote:
> Essentially, pallet lets you use a simple Rakefile stanza (akin to the
> existing gemspec stanza) to define multiple packaging formats. Right now,
> gem and dpkg are the only formats supported, although adding .tar.gz
> should be trivial, and .rpm should be simple enough for anyone who's built
> an RPM.

I'd definitely be interested in RPM.

Right now I'm using a different approach: gem2spec takes a gem and turns
it
into an RPM spec file. Unlike the native .deb packages you're generating
under Pallet, the components are installed under /usr/lib/ruby/gems/1.8
not
/usr/lib/ruby/1.8.

$ rpm -ql rubygem-rake
/usr/bin/rake
/usr/lib/ruby/gems/1.8/cache/rake-0.7.1.gem
/usr/lib/ruby/gems/1.8/doc/rake-0.7.1
/usr/lib/ruby/gems/1.8/doc/rake-0.7.1/rdoc
/usr/lib/ruby/gems/1.8/doc/rake-0.7.1/rdoc/classes
/usr/lib/ruby/gems/1.8/doc/rake-0.7.1/rdoc/classes/CompositePublisher.html
...
/usr/lib/ruby/gems/1.8/doc/rake-0.7.1/ri
/usr/lib/ruby/gems/1.8/doc/rake-0.7.1/ri/CompositePublisher
/usr/lib/ruby/gems/1.8/doc/rake-0.7.1/ri/CompositePublisher/add-i.yaml
...
/usr/lib/ruby/gems/1.8/gems/rake-0.7.1
/usr/lib/ruby/gems/1.8/gems/rake-0.7.1/CHANGES
/usr/lib/ruby/gems/1.8/gems/rake-0.7.1/MIT-LICENSE
/usr/lib/ruby/gems/1.8/gems/rake-0.7.1/README
/usr/lib/ruby/gems/1.8/gems/rake-0.7.1/Rakefile
..
/usr/lib/ruby/gems/1.8/gems/rake-0.7.1/test/test_test_task.rb
/usr/lib/ruby/gems/1.8/specifications/rake-0.7.1.gemspec

With Pallet, you can tell it what files to bundle together to build a
package. But what about the initial fetch/untar/configure/build/install
steps? It would be very attractive to write these once in ruby (or
rake),
but I don't think Pallet provides rules for this. Are there existing
libraries of rake tasks to do these steps?

Regards,

Brian.

P.S. Another tool I've used occasionally is Debian's "alien" (e.g. to
install the oracle-instantclient RPMs on an Ubuntu box). I've never
tried
using alien to convert a .deb to .rpm though.
Fcc5cdf0f0f3e1a3a39c11ed4bf8d5e5?d=identicon&s=25 Stephan Mueller (Guest)
on 2007-05-02 15:55
(Received via mailing list)
Hi Stephen,

* Stephen Touset <stephen@touset.org> [01.05.2007]:

> I've been writing a multiplatform packaging tool for the company I work
> for, dubbed "pallet". I recently got the go-ahead to release it on
> RubyForge under the LGPL, so have spent the past day or so getting it up
> on there.

oh, this is great! I just grabbed it and like it. :)

one small note:

the created debian files contain references to what i think is one of
your personal projects: non-free/damballa (see debian/files for
examples). Maybe this can be customized when you debianize a project.

Thanks a lot for making this public!


Cheers,

- Steph.
38ed06ce1dd79b986574e9c7b7b374f8?d=identicon&s=25 Tomas Pospisek's Mailing Lists (Guest)
on 2007-05-04 10:26
(Received via mailing list)
On Wed, 2 May 2007, Stephen Touset wrote:

> I've been writing a multiplatform packaging tool for the company I work
> for, dubbed "pallet".
[...]
> Right now, gem and dpkg are the only formats supported, although adding
> .tar.gz should be trivial, and .rpm should be simple enough for anyone
> who's built an RPM.

Have you been talking to Debian's Ruby packagers? Mutating this into
something like "rake dpkg", that would automagically turn any ruby gem
into a Debian package would be brilliant :-o
*t

--
9ff064364151151cbce108b869e6e62d?d=identicon&s=25 unknown (Guest)
on 2007-05-04 18:05
(Received via mailing list)
On May 2, 9:54 am, Stephan Mueller <d...@web.de> wrote:
> one small note:
>
> the created debian files contain references to what i think is one of
> your personal projects: non-free/damballa (see debian/files for
> examples). Maybe this can be customized when you debianize a project.

Oops. I still have a little work to do to remove some of the company-
specific assumptions. Thanks for the heads-up. Please let me know if
you find anything else!
9ff064364151151cbce108b869e6e62d?d=identicon&s=25 unknown (Guest)
on 2007-05-04 18:10
(Received via mailing list)
On May 4, 4:25 am, Tomas Pospisek's Mailing Lists <t...@sourcepole.ch>
wrote:
> into a Debian package would be brilliant :-o
I did send an email to the debian-ruby-extras mailing list, but
haven't heard anything back.

I'm about to start writing (time-willing) a Pallet::Gem2Deb module,
which should simply wrap around the Pallet::Deb builder. Basically,
feed it the task to build a gem, and it will handle the whole thing
for you -- including extracting relevant metadata from the Gem and
applying it automagically.
9ff064364151151cbce108b869e6e62d?d=identicon&s=25 Stephen Touset (Guest)
on 2007-05-05 00:45
(Received via mailing list)
On Wed, 02 May 2007 18:46:15 +0900, Brian Candler wrote:
> steps? It would be very attractive to write these once in ruby (or rake),
> but I don't think Pallet provides rules for this. Are there existing
> libraries of rake tasks to do these steps?

Not of rake tasks, but you could easily write a rake task which
downloads
the file to somewhere predictable.
Fcc5cdf0f0f3e1a3a39c11ed4bf8d5e5?d=identicon&s=25 Stephan Mueller (Guest)
on 2007-05-05 18:39
(Received via mailing list)
* stephen@touset.org <stephen@touset.org> [04.05.2007]:

> Oops. I still have a little work to do to remove some of the company-
> specific assumptions. Thanks for the heads-up. Please let me know if
> you find anything else!

i've using pallet now for 3 or 4 projects here and still like it. :) I
am thinking about enhance pallet in a way that allows it to rebuild all
debian package control files on the fly, when the user whishes to do so.
This would allow to get packages with an up-to-date package version etc.
without handediting these files. What do you think about that?

Maybe a simple template engine could be used to handle build these files
on the fly and give the package maintainer full control on this.


Cheers,

Steph
753dcb78b3a3651127665da4bed3c782?d=identicon&s=25 Brian Candler (Guest)
on 2007-05-05 20:21
(Received via mailing list)
On Sat, May 05, 2007 at 07:45:10AM +0900, Stephen Touset wrote:
> > With Pallet, you can tell it what files to bundle together to build a
> > package. But what about the initial fetch/untar/configure/build/install
> > steps? It would be very attractive to write these once in ruby (or rake),
> > but I don't think Pallet provides rules for this. Are there existing
> > libraries of rake tasks to do these steps?
>
> Not of rake tasks, but you could easily write a rake task which downloads
> the file to somewhere predictable.

Sure, but I was thinking of using it as a generic way to package
third-party
software and use it to generate, say, .deb and .rpm

RPM is what I'm more familiar with. It has a 'spec' file which directs
the
build. But it has sensible default macros for:
- untar source into a work directory
- apply patch(es)
- configure
- make
- make install

You can override any of these, but the defaults work in the majority of
cases.

Now, if I had to then package the same software as .deb, I'd have to
write
(and first learn) Debian control files.

If I could specify the package building once - in Ruby - that would be
great.

Regards,

Brian.
9ff064364151151cbce108b869e6e62d?d=identicon&s=25 Stephen Touset (Guest)
on 2007-05-07 16:21
(Received via mailing list)
On Sun, 06 May 2007 03:20:10 +0900, Brian Candler wrote:

> Sure, but I was thinking of using it as a generic way to package third-party
> software and use it to generate, say, .deb and .rpm

That's what I really aim this project to deal with. It'd be exceedingly
nice if upstream developers would take it upon themselves to package
their
software in multiple formats, but barring that it should be easy for a
distro maintainer to repackage it.

Debian isn't so hot on packing gems inside of dpkgs and just having the
dpkg be a frontend for "gem install foo", but I don't personally mind,
and
other Debian-derived distros (read: Ubuntu) might feel differently as
well.

> RPM is what I'm more familiar with. It has a 'spec' file which directs the
> build. But it has sensible default macros for:
> - untar source into a work directory
> - apply patch(es)
> - configure
> - make
> - make install
>
> You can override any of these, but the defaults work in the majority of
> cases.

I was going to try and tackle writing an RPM backend sometime the next
month, but would have to first learn the entire RPM building process. If
you'd like to write a Pallet::RPM class to generate and build a spec
file,
I'd be more than happy to accept it.

Automatically downloading from remote sources, if done at all, would
probably come after I start supporting more package formats and write
some
Pallet::Gem2(Deb|RPM|foo) classes to wrap Gems inside of OS-native
packages.
9ff064364151151cbce108b869e6e62d?d=identicon&s=25 Stephen Touset (Guest)
on 2007-05-07 16:22
(Received via mailing list)
On Sun, 06 May 2007 01:38:06 +0900, Stephan Mueller wrote:

> without handediting these files. What do you think about that?
>
> Maybe a simple template engine could be used to handle build these files
> on the fly and give the package maintainer full control on this.

Glad you like it!

The files are already templated, but I made the decision to leave the
debian/ directory on the filesystem after a build, for the case where a
package maintainer might want to generate the build structure once, then
hand-edit it and maintain it on his own from then on.

In retrospect, that should probably be done through a separate target
(rake packages:perm_deb, perhaps?). As it stands now, just run a "rake
clobber" or "rake clobber:deb" to get rid of the debian directory and
build again from scratch.
753dcb78b3a3651127665da4bed3c782?d=identicon&s=25 Brian Candler (Guest)
on 2007-05-07 18:52
(Received via mailing list)
On Mon, May 07, 2007 at 11:20:04PM +0900, Stephen Touset wrote:
>
> I was going to try and tackle writing an RPM backend sometime the next
> month, but would have to first learn the entire RPM building process. If
> you'd like to write a Pallet::RPM class to generate and build a spec file,
> I'd be more than happy to accept it.
>
> Automatically downloading from remote sources, if done at all, would
> probably come after I start supporting more package formats and write some
> Pallet::Gem2(Deb|RPM|foo) classes to wrap Gems inside of OS-native packages.

OK, so I'm not entirely clear how Pallet works then.

Does it generate "how to" build rules for the native packaging system -
e.g.
spec files for RPM, whatever the equivalent is for Debian
(control.tar.gz)?
And then you use the native packaging system to actually create the
package?

Or does it configure, build and install the package itself, and then
only
instruct the native packaging system to assemble those files into the
finished package?

Regards,

Brian.
9ff064364151151cbce108b869e6e62d?d=identicon&s=25 Stephen Touset (Guest)
on 2007-05-07 23:46
(Received via mailing list)
On Tue, 08 May 2007 01:51:45 +0900, Brian Candler wrote:

> OK, so I'm not entirely clear how Pallet works then.
>
> Does it generate "how to" build rules for the native packaging system - e.g.
> spec files for RPM, whatever the equivalent is for Debian (control.tar.gz)?
> And then you use the native packaging system to actually create the
> package?

Yes. And, for debian, that's a whole slew of files in a "debian/"
directory, plus a self-executing Makefile called "debian/rules". It's
kind of a PITA, actually. :)

> Or does it configure, build and install the package itself, and then
> only instruct the native packaging system to assemble those files into
> the finished package?

No, it only writes the package-specific configuration (config file, slew
of files in some directory, whatever) files, then instructs the package
system to do the whole thing itself.

Arguably, the latter approach would be cleaner from a software design
point of view, and likely where the project will head in the future. For
now, I wanted to get something released and working. Actual Ruby
libraries
to completely generate .deb/.rpm/.ebuild/.foo would be very useful,
though. Sadly, I don't have infinite time, and I'm nowhere near as
productive as _why.
38ed06ce1dd79b986574e9c7b7b374f8?d=identicon&s=25 Tomas Pospisek's Mailing Lists (Guest)
on 2007-05-08 07:41
(Received via mailing list)
On Tue, 8 May 2007, Stephen Touset wrote:

> directory, plus a self-executing Makefile called "debian/rules". It's
> kind of a PITA, actually. :)

Uhm - as far as I know you only need like - two - of them:
debian/control
and debian/rules. The others are optional in various degrees (such as
debian/copyright is reasonable if you want to distribute the package,
isn't it?)
*t

--
9ff064364151151cbce108b869e6e62d?d=identicon&s=25 Stephen Touset (Guest)
on 2007-05-09 16:30
(Received via mailing list)
On Tue, 08 May 2007 14:40:54 +0900, Tomas Pospisek's Mailing Lists
wrote:

>>
>> Yes. And, for debian, that's a whole slew of files in a "debian/"
>> directory, plus a self-executing Makefile called "debian/rules". It's
>> kind of a PITA, actually. :)
>
> Uhm - as far as I know you only need like - two - of them: debian/control
> and debian/rules. The others are optional in various degrees (such as
> debian/copyright is reasonable if you want to distribute the package,
> isn't it?)

Right. Unfortunately, to do lots of things programmatically, you need
plenty more than that in conjunction with debhelper.

  debian/changelog => changelog
  debian/compat    => debhelper compatibility version
  debian/config    => debconf configuration
  debian/control   => package information
  debian/copyright => copyright information
  debian/dirs      => directories that should be created on the target
  debian/docs      => location of documentation
  debian/files     => files to be installed in the target
  debian/rules     => build script

Plus a few others, depending on whether or not a few options are used.

--
Stephen Touset <stephen@touset.org>
38ed06ce1dd79b986574e9c7b7b374f8?d=identicon&s=25 Tomas Pospisek's Mailing Lists (Guest)
on 2007-05-09 22:06
(Received via mailing list)
On Wed, 9 May 2007, Stephen Touset wrote:

>>>> And then you use the native packaging system to actually create the
>
>  debian/files     => files to be installed in the target
>  debian/rules     => build script
>
> Plus a few others, depending on whether or not a few options are used.

Take those easy. debhelper has some 61 helper programms of which most
are
configurable or use arguments *only*. So it's also your choice which way
to go and also *if* to use those at all. Neither of those, except for
debian/control and debian/rules are mandatory. Actually you can even
make
a package without debian/rules, it will be installable if you build it
right however Debian itself will not accept it since it will not be
autobuildable. Please correct me if I'm wrong.
*t

--
This topic is locked and can not be replied to.