Ruby packaging in Debian and Ubuntu: Mythbusting and FAQ

Hi,

As questions on the packages provided in Debian and Ubuntu for Ruby and
Rubygems are often raised here, I thought I would write up a summary of
the situation, and answers to the most frequently asked questions or
myths.

See http://www.lucas-nussbaum.net/blog/?p=566

Feel free to discuss it here, or in the blog comments. (I’m monitoring
both)

  • Lucas

Hi Lucas,

[warning: this turned out to be a loooong email]

thanks for going after this topic, and thanks for caring about Ruby on
Debian! I see that people are already actively writing comments on your
blog, however I prefer the list to easier keep track of things (I’m
doing hard following replies on a flat list like the blog comments).

On 12.09.2010 18:29, Lucas N. wrote:

Feel free to discuss it here, or in the blog comments. (I’m
monitoring both)

I’m also one those who replied recently on ruby-core about the
Ruby/Debian issue, not because I want to nag about it but I also care
about it. I’ve much settled about using “rvm” these days to avoid the
problems I had but I think it’s only fair to the immense amount of work
you’re putting into it when a user like me also provides his share of
it, so I thought about getting back and see what problems I had.

For the following things, I’m doing this on Lenny inside VMWare,
2.6.26-2-amd64 that is. It’s a clean system, i.e. freshly installed and
no ruby anywhere. I made a snapshot so I can go anytime back and test
things really from the start if I’m doing something wrong.

Goal: set up ruby so I can develop with nokogiri (it’s one of the common
use cases for me)

Looking at the available ruby packages (I did with dpkg -l ‘ruby’ )
I went with “sudo aptitude install ruby-full”, it sounded the most
reasonable to me.

After the installation, I also installed “libxml2-dev” because due my
prior knowledge I know what nokogiri is based on it. Install of that
went fine and then I executed “gem list --remote” because I didn’t see
the nokogiri package on the previous package list:

$ gem list --remote
bash: gem: command not found

So that was the first little wall. I truly was not aware that gem is not
part of ruby on Debian. Or in general? I don’t know that.

Anyway, I cleverly typed dpkg -l ‘gem’ to find it, but was
surprised:

rc imagemagick
un imagemagick-doc
rc kimagemapeditor
un migemo

Nothing about ruby and gems anywhere. I checked my sources.list :

deb http://ftp.at.debian.org/debian/ lenny main contrib non-free
deb-src http://ftp.at.debian.org/debian/ lenny main contrib non-free

deb http://security.debian.org/ lenny/updates main contrib non-free
deb-src http://security.debian.org/ lenny/updates main contrib non-free

deb http://volatile.debian.org/debian-volatile lenny/volatile main
contrib non-free
deb-src http://volatile.debian.org/debian-volatile lenny/volatile main
contrib non-free

Looks pretty complete to me. I went on to
http://www.debian.org/distrib/packages and searched in “Search the
contents of packages” for it, I tried with “bin/gem” in “stable” first
and it came up with “jruby-1.0” and “netbeans-ide”; weird I thought. I
tried again with “gem” but the result was equally lacking what I was
looking for.

Next try was to search “Search package directories” in “stable” for
“gem” and finally at the end I saw “rubygems” and related packages. So I
installed with “sudo aptitude install rubygems”.

This was very puzzling that gem was not part of ruby or even that
ruby-full didn’t not contain the rubygems (which due it’s name implied
it to me).

Next thing: find the package, I ran “gem list --remote”. I got an awful
long list which went by so I need to grep for it to see what’s really
there.

$ gem list --remote|grep nokogiri
ERROR: While executing gem … (Gem::RemoteFetcher::FetchError)
bad response Found 302 (http://gems.rubyforge.org/latest_specs.4.8)

Now that was weird. I ran the previous comment again:

$ gem list --remote

*** REMOTE GEMS ***

ERROR: While executing gem … (Gem::RemoteFetcher::FetchError)
bad response Found 302 (http://gems.rubyforge.org/latest_specs.4.8)

Still not good. I vaguely remembered that “gem” is using user prefs for
it’s things and I discovered ~/.gem/ in my homedir. I removed it and now
the remote listing did work; but only once:

[email protected]:~$ rm -Rf .gem/
[email protected]:~$ gem list --remote|grep nokogiri
glebm-nokogiri (1.4.2.1)
jwagener-nokogiri (1.4.1)
nokogiri (1.4.3.1, 1.2.3)
nokogiri-happymapper (0.3.4)
nokogiri-plist (0.3.1)
revo-nokogiri (1.4.1)
rsolr-nokogiri (0.0.0)
rubyjedi-nokogiri_java (1.4.0.20100513161003)
superfeedr-nokogiri (1.4.0.20091116183308)
[email protected]:~$ gem list --remote|grep nokogiri
ERROR: While executing gem … (Gem::RemoteFetcher::FetchError)
bad response Found 302 (http://gems.rubyforge.org/latest_specs.4.8)

Oh my. So gem installation wouldn’t work either unless I removed ~/.gem
again:

$ gem list --remote|grep nokogiri
ERROR: While executing gem … (Gem::RemoteFetcher::FetchError)
bad response Found 302 (http://gems.rubyforge.org/latest_specs.4.8)
$ gem install nokogiri
ERROR: could not find gem nokogiri locally or in a repository
$ rm -Rf .gem/
$ gem install nokogiri
ERROR: While executing gem … (Errno::ENOENT)
No such file or directory -
/var/lib/gems/1.8/cache/nokogiri-1.4.3.1.gem

Or so I thought …

At that point I lost interest in getting things up and running (I tried
it twice from a clean VM snapshot). I reverted to clean snapshot and
went for the “rvm” way, starting at
http://rvm.beginrescueend.com/rvm/install/ .

I took me a while to figure out what packages I’m required to have to
get “rvm” properly running, which meant I had re-run the rvm
installation process multiple times until I got through. That took my
about ten minutes (and it was not very fun); in the end I installed
“make”, “gcc”, “bzip2”, “git-core”, “zlib1g-dev” and “curl” and added
this line to end of .bashrc

[[ -s “$HOME/.rvm/scripts/rvm” ]] && source “$HOME/.rvm/scripts/rvm”

and started a new shell and installed 1.8.7 (the version used as the
default package in Debian):

$ rvm install 1.8.7
[…]
$ rvm use 1.8.7
info: Using /home/mfischer/.rvm/gems/ruby-1.8.7-p302

After that I installed libxml2-dev

$ gem install nokogiri

and I quickly discovered that I also need libxslt-dev to continue. This
is how the final installation looked:

$ gem install nokogiri
Building native extensions. This could take a while…
Successfully installed nokogiri-1.4.3.1
1 gem installed
Installing ri documentation for nokogiri-1.4.3.1…

No definition for parse_memory

No definition for parse_file

No definition for parse_with

No definition for get_options

No definition for set_options
Installing RDoc documentation for nokogiri-1.4.3.1…

No definition for parse_memory

No definition for parse_file

No definition for parse_with

No definition for get_options

No definition for set_options

No idea what those “No definition” stuff is about,.

$ cat - > test.rb
require ‘rubygems’
require ‘nokogiri’
p Nokogiri::HTML(“

Mr. Belvedere Fan
Club

”)
$ ruby test.rb
#<Nokogiri::HTML::Document:0x3fd62103b114 name=“document”
children=[#<Nokogiri::XML::DTD:0x3fd62103aed0 name=“html”>,
#<Nokogiri::XML::Element:0x3fd62103ae80 name=“html”
children=[#<Nokogiri::XML::Element:0x3fd62103a764 name=“body”
children=[#<Nokogiri::XML::Element:0x3fd62103a4e4 name=“h1”
children=[#<Nokogiri::XML::Text:0x3fd62103a264 “Mr. Belvedere Fan
Club”>]>]>]>]>

Installation worked.

Although getting “rvm” up and running wasn’t a piece of cake either, but
the problems I faced were purely related to install missing packages,
because the errors very obvious to me. But that was not the case why I
could run “gem list --remote” only the first time successfully. Package
error? Ruby error? I wouldn’t know, I’m not that an expert.

Collecting the missing packages for getting “rvm” up was cumbersome, in
recall from the past that usually many of them are installed already
(most notably the compiling chain) on the systems I work one.

Next I tried if I can get further with the Ruby 1.9 packages:

$ sudo aptitude install ruby1.9-full rubygems1.9 libxml2-dev libxslt-dev

I typed those out of my mind because I thought that would be the equal
package names, but I was wrong. There’s no full for 1.9 on Lenny, this
was the proper line:

$ sudo aptitude install ruby1.9 ruby1.9-dev rubygems1.9 libxml2-dev
libxslt-dev

However, running “gem list --remote” twice didn’t work either:

$ rm -Rf .gem ; gem list --remote|grep nokogiri ; echo
“------------------” ; gem list --remote|grep nokogiri
glebm-nokogiri (1.4.2.1)
jwagener-nokogiri (1.4.1)
nokogiri (1.4.3.1, 1.2.3)
nokogiri-happymapper (0.3.4)
nokogiri-plist (0.3.1)
revo-nokogiri (1.4.1)
rsolr-nokogiri (0.0.0)
rubyjedi-nokogiri_java (1.4.0.20100513161003)
superfeedr-nokogiri (1.4.0.20091116183308)

ERROR: While executing gem … (Gem::RemoteFetcher::FetchError)
bad response Found 302 (http://gems.rubyforge.org/latest_specs.4.8)

I’ve no idea what’s going on here.

Now as to why I didn’t report a bug at Debian yet: if I encountered the
error in the past (which I don’t remember to be honestly) it would have
been during working-hours and depending on my available time there I may
have not had the time to report the bug. Now that I could reproduce it,
shall I create one? I’m not sure where the bug exactly is here…

Lucas, thanks for the initiative and for listening. I hope I could help
with that mail in anyway.

sincerely,

  • Markus

Hi,

On 12.09.2010 23:05, Lucas N. wrote:

Anyway, I cleverly typed dpkg -l ‘gem’ to find it, but was surprised:

dpkg -l lists locally installed packages. You want apt-cache search.

Ok thanks, wasn’t aware of that!

I guess that’s because the version of rubygems in lenny still users
rubyforge. The default location for gems was moved two times since then.

[…]

It would actually be more interesting if you could report on your
experience doing the same thing with Debian squeeze (the next release,
currently frozen). There’s not much we can do about lenny anyway.

Hmm, I understand. Well, squeeze … I don’t recall any of the machines
I use (at work, that’s my main reason to use Ruby) runs on Squeeze; all
production and thus environment machines are on Lenny.

But given your statement, it’s one more reason to use “rvm” …

  • Markus

On 13/09/10 at 05:37 +0900, Markus F. wrote:

Feel free to discuss it here, or in the blog comments. (I’m
2.6.26-2-amd64 that is. It’s a clean system, i.e. freshly installed and
After the installation, I also installed “libxml2-dev” because due my
prior knowledge I know what nokogiri is based on it. Install of that
went fine and then I executed “gem list --remote” because I didn’t see
the nokogiri package on the previous package list:

$ gem list --remote
bash: gem: command not found

So that was the first little wall. I truly was not aware that gem is not
part of ruby on Debian. Or in general? I don’t know that.

It’s not part of Ruby 1.8.

Anyway, I cleverly typed dpkg -l ‘gem’ to find it, but was surprised:

dpkg -l lists locally installed packages. You want apt-cache search.

deb http://security.debian.org/ lenny/updates main contrib non-free
and it came up with “jruby-1.0” and “netbeans-ide”; weird I thought. I

Now that was weird. I ran the previous comment again:

$ gem list --remote

*** REMOTE GEMS ***

ERROR: While executing gem … (Gem::RemoteFetcher::FetchError)
bad response Found 302 (http://gems.rubyforge.org/latest_specs.4.8)

I guess that’s because the version of rubygems in lenny still users
rubyforge. The default location for gems was moved two times since then.

[…]

Next I tried if I can get further with the Ruby 1.9 packages:

$ sudo aptitude install ruby1.9-full rubygems1.9 libxml2-dev libxslt-dev

I typed those out of my mind because I thought that would be the equal
package names, but I was wrong. There’s no full for 1.9 on Lenny, this
was the proper line:

Indeed, it was added later.

$ sudo aptitude install ruby1.9 ruby1.9-dev rubygems1.9 libxml2-dev
libxslt-dev

nokogiri-plist (0.3.1)

Now as to why I didn’t report a bug at Debian yet: if I encountered the
error in the past (which I don’t remember to be honestly) it would have
been during working-hours and depending on my available time there I may
have not had the time to report the bug. Now that I could reproduce it,
shall I create one? I’m not sure where the bug exactly is here…

Lucas, thanks for the initiative and for listening. I hope I could help
with that mail in anyway.

It would actually be more interesting if you could report on your
experience doing the same thing with Debian squeeze (the next release,
currently frozen). There’s not much we can do about lenny anyway.

Thanks for the feedback,

  • Lucas

On Mon, 13 Sep 2010 01:29:21 +0900, Lucas N. wrote:

Hi,

As questions on the packages provided in Debian and Ubuntu for Ruby and
Rubygems are often raised here, I thought I would write up a summary of
the situation, and answers to the most frequently asked questions or
myths.

See http://www.lucas-nussbaum.net/blog/?p=566

  • Myth: Ruby is completely outdated in Debian and Ubuntu

It’s a bit of a myth that the culture in the Ruby community is to always
use the latest version. People were scared of Ruby 1.8.7 for a long time
because it changed APIs dramatically. (I actually think Debian unstable
was closer to the leading edge of adopting Ruby 1.8.7.) I doubt that
1.9.x adoption is anywhere near 1.8.x adoption at this point.

I think that the Ruby community is aware of both of these trends, and I
think that’s why somebody wrote RVM in the first place. (I don’t use
RVM,
somehow I suspect it would badly break my Debian.)

Ruby’s adoption of an API version number for Ruby 1.9.x (which you are
aware of – it’s reflected in your Debian packaging) is an
acknolwedgement that APIs shouldn’t be changed in breaking ways when so
many people depend on them.

I personally think you’re doing the right thing with Ruby versions. I
just wish that I didn’t have to remember to type irb1.9.1 all the time
for a Ruby 1.9 interpreter. I had gotten used to having only 2 digits.
Maybe for Ruby 2.0 (whenver it happens) they can stop making API changes
at the x.x.1 level, and make new x.1 releases when they feel the need to
change the API.

If there’s enough demand for it (and it isn’t too much work), maybe a
ruby-snapshot package is worthwhile, like gcc-snapshot and
llvm-snapshot.

  • Myth: Rubygems is crippled in Debian

I think that allowing gem update --system is a bad idea. (Then again,
disabling gem update --system was my request to begin with, #452547 if
you recall.) If someone needs a more recent Rubygems vesion, they should
go for backports.

By the way, the verison of Rubygems in lenny is quite outdated – all of
the gems in the archive now have a different way of expressing
dependencies that the old version in lenny is incompatible with. I’m
guessing that’s the gemcutter thing that raggi is talking about in the
comments to your blog. I had to get a backport for the lenny systems I
administer. Any chance this can be upgraded in a lenny point release, or
is lenny basically obsolete anyway as soon as squeeze is out?

It would actaually be pretty nice if people could tell us what advantage
they would get from running gem update --system on a Debian system,
aside
from that warm fuzzy feeling of having the latest and greatest.

The thing that I think Rubygems does need is an “equivs” sort of
feature. You’ve included lots of good ruby libraries in Debian’s
archive,
but rubygems doesn’t know about them. If I want to install a rubygem
that
depends on something that I’ve already installed from Debian’s archive,
rubygems pulls in another copy of that library. You and the Rubygems
people need to develop a way of giving Rubygems metadata about libraries
that are already installed on the system externally to Rubygems. That
way, Rubygems can use that information for dependancy resolution. (So I
think of this as having some sort of /usr/share/rubygems/externals.d
which gets filled with YAML files listing the names of gems that a
particular package provides.)

Also, I have Debian bug #529663 “Gem::Ext::RakeBuilder uses the wrong
executable name for rake” outstanding for over a year. Has this been
fixed?

  • Why doesn’t Ruby use the alternatives system on Debian?

I just want to add to this (for the Ruby community to hear) that we
treat
GCC and Python versions the same way in Debian. There is a default
version of the language, and if you want anything other than the default
version of the language, you need to specify that version in the shebang
line. Changing the symlink yourself can break things.

Changing shebangs from /usr/bin/ruby to /usr/bin/ruby1.8 should be a
release goal in wheezy.

  • /usr/local

I’m not sure to what extent the Ruby packagers have been looking at how
Python and Perl solve the same sorts of issues (Python with its version
numbers, and Perl with CPAN), but it’s probably worth adopting
approaches
from both. Looking at how CPAN puts all of its stuff in /usr/local,
that’s probably the right approach for Rubygems.

Commenting on the comments now…

Raggi wrote “To be quite honest, the comment about smartphones and
embedded systems is really an odd one.” Debian developers tend to think
about packaging in terms of how to break up a package into managable
chunks that can be used on small(ish) embedded systems. Think Nokia
770s,
Marvel Sheevaplugs, and Linksys NSLU2s and the like. The packaging level
is usually where Debian’s share of the responsibility for embedded
systems ends. From there, it’s up to the upstream developers to be
embedded-systems friendly. Also, when Debian’s packages this way, it can
work as a good starting point for further tuning for embedded systems
development.

Between RVM, Ruby 1.8.7 (and other such API breakage), and the big
Gemcutter changes, I think that the Ruby developers have a lot to learn
about good release engineering, and the Ruby community is getting
militant at Debian, because Debian can’t satisfy compatibility under
the circumstances Ruby developers give them. An in-person summit between
the Ruby core team, and the distributions that package Ruby (and maybe a
Python packager and Perl packager to get their expertise on how things
are done and why) might go a long way towards building sustainable
release practices on all sides. (I can’t imagine that other
distributions
don’t have some of the same issues.)

The Ruby community needs to get used to distributions packaging their
software in a way that is compatible across versions.

–Ken B.
(A mostly happy user of Debian’s packaged Ruby and Rubygems.)

On 13/09/10 at 07:43 +0900, Ken B. wrote:

I personally think you’re doing the right thing with Ruby versions. I
just wish that I didn’t have to remember to type irb1.9.1 all the time
for a Ruby 1.9 interpreter. I had gotten used to having only 2 digits.
Maybe for Ruby 2.0 (whenver it happens) they can stop making API changes
at the x.x.1 level, and make new x.1 releases when they feel the need to
change the API.

Yeah, I agree that the “1.9.1” suffix is a bit painful, but I don’t
think that there is a better solution. We plan to investigate the switch
to the “alternatives” system in Debian, but that’s a lot of work.

If there’s enough demand for it (and it isn’t too much work), maybe a
ruby-snapshot package is worthwhile, like gcc-snapshot and llvm-snapshot.

We usually provide packages for the development branch. We have been
providing a 1.9 package since 2005, so we are not too far from a
ruby-snapshot package. Now that 1.9.2 is officially stable, we might
think about providing a 1.9.3 snapshot package. But I don’t think that
we have the manpower to provide packages for 1.8.7, 1.9.2 and 1.9.3.

By the way, the verison of Rubygems in lenny is quite outdated – all of
the gems in the archive now have a different way of expressing
dependencies that the old version in lenny is incompatible with. I’m
guessing that’s the gemcutter thing that raggi is talking about in the
comments to your blog. I had to get a backport for the lenny systems I
administer. Any chance this can be upgraded in a lenny point release, or
is lenny basically obsolete anyway as soon as squeeze is out?

lenny will be supported for one year after the squeeze release. But I
don’t think that it is reasonable to update rubygems in lenny with a
newer version. The fact that the ruby package manager itself changed in
incompatible ways is really a problem.
The best solution would be to provide backports for the main Ruby
packages (interpreter + rubygems, probably) on Debian and Ubuntu. If
someone wants to work on that, that is quite easy to do. And it would be
wonderful to offload that task to someone.

that are already installed on the system externally to Rubygems. That
way, Rubygems can use that information for dependancy resolution. (So I
think of this as having some sort of /usr/share/rubygems/externals.d
which gets filled with YAML files listing the names of gems that a
particular package provides.)

Also, I have Debian bug #529663 “Gem::Ext::RakeBuilder uses the wrong
executable name for rake” outstanding for over a year. Has this been
fixed?

Not yet. Needs someone (you? :slight_smile: to investigate this, and propose a
solution. Unfortunately, my own plate is full. :frowning:

  • Lucas

On 13/09/10 at 06:19 +0900, Markus F. wrote:

rubyforge. The default location for gems was moved two times since then.

But given your statement, it’s one more reason to use “rvm” …

You can see things differently. Lenny was released in february 2009.
People have installed on servers after that, and planned at the same to
use it for up to three years. During this three years lifetime, they
expect it to stay completely stable (only important bugs, like the
security issues, are supposed to be fixed). That’s what Debian provides
with its stable releases.

Of course, if it doesn’t solve your problem, you should try RVM, or
backports (but we don’t provide any currently due to lack of time,
unfortunately).

  • Lucas

On Sun, Sep 12, 2010 at 4:37 PM, Markus F. [email protected]
wrote:

After that I installed libxml2-dev

$ gem install nokogiri

and I quickly discovered that I also need libxslt-dev to continue.

Just a note that http://nokogiri.org/tutorials/installing_nokogiri.html
has
a section for Debian/Ubuntu with instructions that include the command:

sudo apt-get install libxslt-dev libxml2-dev

:slight_smile:

These are rdoc warnings. They can be ignored at gem-install time.

Lucas, thanks for all your work maintaining ruby on Debian!

It seems like you’re doing a good job; most of the things that feel
“broken” to me sound like they will be fixed in the next version. The
only other suggestion I have is this: make the “ruby” package install
the whole shebang: irb, ri, rdoc, libwhatever, rubygems… Then you
could have a “ruby-minimal” package for the tiny minority of people
doing embedded systems work in ruby. Doing this would save a great deal
of newbie frustration, and cut down on complaints here and elsewhere.

Software in general is frustrating, and frustrated people tend to be
rude. Don’t let it get to you!

On Mon, Sep 13, 2010 at 4:13 AM, Ken B. [email protected] wrote:

I think that the Ruby community is aware of both of these trends, and I
think that’s why somebody wrote RVM in the first place. (I don’t use RVM,
somehow I suspect it would badly break my Debian.)

One of the design features of RVM is that it installs everything as a
regular user, in /home/whoever, therefore it cannot break your
Debian. For what it’s worth I’m using it very happily both at home
under arch linux and at work under ubuntu and have had zero problems.

martin

“Lucas N.” [email protected] schreef in bericht
news:[email protected]

both)

  • Lucas

Last week I struggled a lot with Ruby, Rubygems and Rails in Ubuntu
10.04.
Till I found the Bitnami Rubystack on the internet.
Installed it and its works perfect!!!

http://bitnami.org/stack/rubystack

Marcel

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs