Gem list -r -d * generates Error* on http://gems.github.com

john_old · July 31, 2009, 9:02pm

I’m using ruby 1.8.6 & gem 1.3.5 :

I receive and error trying to list the details of

C:\Users\spanton>gem source
*** CURRENT SOURCES ***

http://gems.github.com

C:\Users\spanton>gem list -r -d

*** REMOTE GEMS ***

ERROR: While executing gem … (ArgumentError)
comparison of String with 3 failed

Error also occurs, when using gem list -r -d a

Yet it works ok, when using gem list -r -d b

Any ideas or workaround?

Thanks for the help.
John

john_old · July 31, 2009, 10:00pm

On Jul 31, 3:57 pm, John [email protected] wrote:

Please provide the output of gem with --debug and -V options enabled
for us to be able to help you.

john_old · August 2, 2009, 5:45pm

On Jul 31, 3:49 pm, Eric H. [email protected] wrote:

C:\Users\spanton>gem source

$ gem list -r -d --sourcehttp://gems.github.com-V andreasronge-neo4j

comparison of String with 3 failed (ArgumentError)
end; Marshal.load File.read(“neo4j.gemspec”)’
286 # Load custom marshal format, re-initializing defaults as
→ 296 field_count = if spec.specification_version >
305 raise TypeError, "invalid Gem::Specification format
array[3]
commit 0e10242e2ef63addf3ac294bb2e932202fa900fc
 released as 0.0.3. Updated documentation
s.bindir = “bin”
s.name = “neo4j”
require_paths.
value = spec.instance_variable_get(ivar)
result << " s.#{name} = #{value}\n"
of Gem::Specification#to_ruby which had RDoc all the way back to 0.8.11!

So if you own one of these, please delete that code!

Fortunately, I now have new things to protect gem repositories from!
(Ryan hit me for not protecting RubyGems enough)

And again, a gemspec is not something that you just dump something on.
It’s not a big truck.- Hide quoted text -

Show quoted text -

Is there a workaround to this problem, which would allow me to
obtain the name & description for gems starting with “a”.

Thanks,
John

john_old · August 1, 2009, 1:18am

On Jul 31, 2009, at 12:00, John wrote:

*** REMOTE GEMS ***

ERROR: While executing gem … (ArgumentError)
comparison of String with 3 failed

Error also occurs, when using gem list -r -d a

Yet it works ok, when using gem list -r -d b

Any ideas or workaround?

$ rm -r ~/.gem/specs/gems.github.com%80/

I did this with just a as the search first and hit this gem:

$ gem list -r -d --source http://gems.github.com -V andreasronge-neo4j

*** REMOTE GEMS ***

GET 200 OK: http://gems.github.com/latest_specs.4.8.gz
GET 200 OK:
http://gems.github.com/quick/Marshal.4.8/andreasronge-neo4j-0.0.4.gemspec.rz
ERROR: While executing gem … (ArgumentError)
comparison of String with 3 failed

So removing this gem from the index and rebuilding should fix the
problem.

Anyhow, let’s see why it’s broken:

$ curl
http://gems.github.com/quick/Marshal.4.8/andreasronge-neo4j-0.0.4.gemspec.rz

neo4j.gemspec.rz

Let’s dump it out:

$ ruby -rubygems -e ‘Marshal.load
Gem.inflate(File.read(“neo4j.gemspec.rz”))’
/Library/Ruby/Site/1.8/rubygems/specification.rb:296:in >': comparison of String with 3 failed (ArgumentError) from /Library/Ruby/Site/1.8/rubygems/specification.rb:296:in _load’
from -e:1:in `load’
from -e:1

Yup, still fails, heavy artillery time! Need to fake out the
Gem::Specification stuff and dump the spec:

$ ruby -e ‘module Gem; class Requirement; def marshal_load(arr) @arr =
arr; end; def inspect() @arr end; end; Version = Requirement; class
Specification; def self._load(str) p Marshal.load(str); nil end end;
end; Marshal.load File.read(“neo4j.gemspec”)’
[“1.2.0”, “2”, “andreasronge-neo4j”, 0.0.4, Wed Oct 22 15:00:00 -0700
2008, “A graph database for JRuby”, >=#Gem::Requirement:0x34c60,

=#Gem::Requirement:0x34b98, “ruby”, [], “neo4j”,
"[email protected]
", [“Andreas R.”], “A graph database for JRuby”,
"http://github.com/andreasronge/neo4j/tree
", “true”, “ruby”, nil]

Ok, so now we have a dump of the gem. Let’s look at where the error
came from in Gem::Specification:

285    ##
286    # Load custom marshal format, re-initializing defaults as

needed
287
288 def self._load(str)
289 array = Marshal.load str
290
291 spec = Gem::Specification.new
292 spec.instance_variable_set :@specification_version,
array[1]
293
294 current_version = CURRENT_SPECIFICATION_VERSION
295
→ 296 field_count = if spec.specification_version >
current_version then
297
spec.instance_variable_set :@specification_version,
298 current_version
299 MARSHAL_FIELDS[current_version]
300 else
301 MARSHAL_FIELDS[spec.specification_version]
302 end
303
304 if array.size < field_count then
305 raise TypeError, “invalid Gem::Specification format
#{array.inspec
t}”
306 end
307
308 spec.instance_variable_set :@rubygems_version,
array[0]
309 # spec version
310 spec.instance_variable_set :@name,
array[2]
311 spec.instance_variable_set :@version,
array[3]

So, specification_version is what’s broken with the gem. RubyGems
expects it to be a number, but we can clearly see in the above dump
that the second item is a String. The version of the gem is stored in
the fourth field, 0.0.4

Pulling down this project and looking through the history we find:

$ git log --grep=release
[…]
commit 0e10242e2ef63addf3ac294bb2e932202fa900fc
Author: Andreas R. [email protected]
Date: Mon Nov 17 23:05:26 2008 +0100

 release 0.0.5, added CHANGELOG

commit 0b283793ee2c8e0689274d85f049bb8c2cf4250c
Author: andreas [email protected]
Date: Thu Oct 23 16:19:27 2008 +0200

 released to rubyforge gem repository

commit 22bd9de5a5a6874ec4903af60fb5a95606f229ee
Author: andreas [email protected]
Date: Thu Oct 23 10:38:33 2008 +0200

 prepare for rubyforge release of gem. Added license file

commit 715e4384f0cc34a56580c2c52054e2756c5d06ac
Author: andreas [email protected]
Date: Mon Oct 20 12:13:00 2008 +0200

 released as 0.0.3. Updated documentation

So it looks like 0b283793ee2c8e0689274d85f049bb8c2cf4250c is release
0.0.4.

$ git checkout 0b283793
$ cat neo4j.gemspec

WARNING : RAKE AUTO-GENERATED FILE. DO NOT MANUALLY EDIT!

LAST UPDATED : Thu Oct 23 15:05:22 +0200 2008

RUN : ‘rake gem:update_gemspec’

Gem::Specification.new do |s|
s.description = “A graph database for JRuby”
s.files = [

…

]
s.rubygems_version = “1.2.0”
s.platform = “ruby”
s.date = “Thu Oct 23 00:00:00 +0200 2008” # ← STRING?
s.homepage = “http://github.com/andreasronge/neo4j/tree”
s.rubyforge_project = “neo4j”
s.bindir = “bin”
s.summary = “A graph database for JRuby”
s.rdoc_options = […]
s.specification_version = “2” # ← STRING?
s.email = “[email protected]”
s.version = “0.0.4”
s.required_rubygems_version = “>= 0”
s.require_paths = [“lib”]
s.required_ruby_version = “>= 1.8.4”
s.extra_rdoc_files = [“README.rdoc”]
s.has_rdoc = “true”
s.name = “neo4j”
s.authors = [“Andreas R.”]
s.loaded = “false”
end

So the gemspec has a String for specification_version. Also, why is
the date a String? It should be a Time object! It seems that this
was hand-built.

There’s a bunch of extra fields set here that RubyGems has defaults
for, like required_rubygems_version, required_ruby_version,
require_paths.

Hrm, what’s that Rake task look like?

$ cat Rakefile

[…]

Thanks to the Merb project for this code.

desc “Update Github Gemspec”
task :update_gemspec do

[…]

 spec.instance_variables.each do |ivar|
   value = spec.instance_variable_get(ivar)
   name = ivar.split("@").last

[…]

     case value
     when Array
       value = # ...
     when String
       value = # ...
     else
       value = value.to_s.inspect # WHAT?
     end
     result << " s.#{name} = #{value}\n"
   end
 end
 result << "end"
 File.open(File.join(File.dirname(__FILE__),

“#{spec.name}.gemspec”), “w”){|f
| f << result}
end

So the author of this gem isn’t entirely to blame here. What’s
especially strange is that this is an especially broken implementation
of Gem::Specification#to_ruby which had RDoc all the way back to 0.8.11!

Looking in the merb source, it seems this code was added here:

commit 7d14e7e28be657c9a1a6b4f0730dd4f6d2f7031b
Author: Daniel Neighman [email protected]
Date: Fri Apr 25 13:07:49 2008 +1000

 Adds Code for generating a gemspec for github

So it looks like Daniel Neighman is to blame. (He may not actually
be, since he doesn’t say where he got this code from.)

I’m not sure how this code was written without looking through
Gem::Specification, nor how the author missed
Gem::Specification#to_ruby (which is exactly what they wanted).

It appears that there are other projects infected by this code:

http://www.google.com/codesearch?q=skip_fields+new_platform+original_platform

So if you own one of these, please delete that code!

Fortunately, I now have new things to protect gem repositories from!
(Ryan hit me for not protecting RubyGems enough)

And again, a gemspec is not something that you just dump something on.
It’s not a big truck.

john_old · August 3, 2009, 7:07am

On Aug 2, 2009, at 08:45 , John wrote: