Forum: Ruby Expanding gemspecs

Hi everyone, I've got a question about the existence or potential
usefulness of a particular gem.

The situation is this: all gems have a gemspec file that defines various
properties about the gem. When creating a new gem with bundler, it puts
in some shortcuts to generating that information, such as using git to
list files in the gem, requiring a file and class constant to retrieve
the version number and using grep to discover test files.

This is great during development, but has a slight performance effect
for the end user when loading the gem. Multiply that by 20, 30, 40+ and
you start to get a noticeable delay when loading all gems (particularly
noticeable when starting rails).

So far I haven't said anything new: this has been brought up before and,
as far as I know, there isn't a universal solution yet. Some people
create Rake tasks to build their gemspecs, but what about a gem? E.g. a
command line tool to produce a fully qualified gemspec from a template.
I think that Jeweller used to do it, but that's no longer maintained as
people tend towards using bundler now.

Is there a gem in existence that parses the gemspec, producing an
expanded version that can then be used when packaging and delivering the
gem? A quick Google search returned nothing, so I wrote a small script
to do it. I was toying with the idea of turning it into a gem, but
wanted to pass it under the noses of seasoned rubyists to see what the
general feeling was.

Any thoughts?

Cheers, Jon :)

On Wednesday, February 6, 2013 4:48:36 AM UTC-5, Jon Cairns wrote:
> This is great during development, but has a slight performance effect
>
>
I am glad you asked.

https://github.com/rubyworks/indexer

On Feb 6, 2013, at 1:48, Jon Cairns <lists@ruby-forum.com> wrote:

> The situation is this: all gems have a gemspec file that defines various
> properties about the gem. When creating a new gem with bundler, it puts
> in some shortcuts to generating that information, such as using git to
> list files in the gem, requiring a file and class constant to retrieve
> the version number and using grep to discover test files.
>
> This is great during development, but has a slight performance effect
> for the end user when loading the gem. Multiply that by 20, 30, 40+ and
> you start to get a noticeable delay when loading all gems (particularly
> noticeable when starting rails).

Gems are packaged with static data only. You can see that be looking at 
any of the specifications in your gem home. Am I missing something?

> Gems are packaged with static data only.

That doesn't seem to be the case, at least for my gems! It appears that
it's down to the author. Here's a random example I picked from my gem
home, highline-1.6.15:

---------
SPEC = Gem::Specification.new do |spec|
  spec.name     = "highline"
  spec.version  = GEM_VERSION
  spec.platform = Gem::Platform::RUBY
  spec.summary  = "HighLine is a high-level command-line IO library."
  spec.files    = `git ls-files`.split("\n")

  spec.test_files       =  `git ls-files -- test/*.rb`.split("\n")
  spec.has_rdoc         =  true
  spec.extra_rdoc_files =  %w[README.rdoc INSTALL TODO CHANGELOG
LICENSE]
  spec.rdoc_options     << '--title' << 'HighLine Documentation' <<
                           '--main'  << 'README'

  spec.require_path      = 'lib'

  spec.author            = "James Edward Gray II"
  spec.email             = "james@graysoftinc.com"
  spec.rubyforge_project = "highline"
  spec.homepage          = "http://highline.rubyforge.org"
  spec.description       = <<END_DESC
A high-level IO library that provides validation, type conversion, and
more for
command-line interfaces. HighLine also includes a complete menu system
that can
crank out anything from simple list selection to complete shells with
just
minutes of work.
END_DESC
end
----------

As you can see, there's a `git ls...` going on in there, and I've seen
plenty of gems that do something similar.

> https://github.com/rubyworks/indexer

That looks like a really interesting project. The only reason I'd say
that it doesn't quite fit this case is that it requires a separate
DSL/syntax/file, rather than the user being able to build a gemspec as a
gemspec, and expand the dynamically generated information as part of the
build.

It would be nice if it could be integrated into `gem build` in some way,
but I'd settle with a separate step for the time being.

On Feb 6, 2013, at 02:53 , Jon Cairns <lists@ruby-forum.com> wrote:

>  spec.files    = `git ls-files`.split("\n")
>
>  spec.test_files       =  `git ls-files -- test/*.rb`.split("\n")

You're looking at a plain old file sitting in the gem bundle. While 
that's used for packaging, it isn't the gem's specification. Look in 
$(gem env gemdir)/specifications:

% gem i -i xxx highline
Fetching: highline-1.6.15.gem (100%)
Successfully installed highline-1.6.15
1 gem installed

% cat xxx/specifications/highline-1.6.15.gemspec
# -*- encoding: utf-8 -*-

Gem::Specification.new do |s|
  s.name = "highline"
  s.version = "1.6.15"

  s.required_rubygems_version = Gem::Requirement.new(">= 0") if 
s.respond_to? :required_rubygems_version=
  s.authors = ["James Edward Gray II"]
  s.date = "2012-09-13"
  s.description = "A high-level IO library that provides validation, 
type conversion, and more for\ncommand-line interfaces. HighLine also 
includes a complete menu system that can\ncrank out anything from simple 
list selection to complete shells with just\nminutes of work.\n"
  s.email = "james@graysoftinc.com"
  s.extra_rdoc_files = ["README.rdoc", "INSTALL", "TODO", "CHANGELOG", 
"LICENSE"]
  s.files = ["README.rdoc", "INSTALL", "TODO", "CHANGELOG", "LICENSE"]
  s.homepage = "http://highline.rubyforge.org"
  s.rdoc_options = ["--title", "HighLine Documentation", "--main", 
"README"]
  s.require_paths = ["lib"]
  s.rubyforge_project = "highline"
  s.rubygems_version = "1.8.25"
  s.summary = "HighLine is a high-level command-line IO library."

  if s.respond_to? :specification_version then
    s.specification_version = 3

    if Gem::Version.new(Gem::VERSION) >= Gem::Version.new('1.2.0') then
    else
    end
  else
  end
end

> You're looking at a plain old file sitting in the gem bundle. While
that's used for packaging, it isn't the gem's specification. Look in 
$(gem
env gemdir)/specifications:

Ah, so the gemspec file in the gem directory is not loaded when loading 
the
gem itself? That makes sense. You learn something every day!

It looks like this isn't an issue then, as gem already does what I was
hoping it would do.

Thanks Ryan