Ruby test suites?

Ruby
1

Has anyone got an overview of the current status of the various Ruby
spec suites? I know of at least 3 (rubicon, jRuby’s tests, and
rubinius’ specs), but I’ve got no idea how complete or active they are,
or which Ruby version they’re attempting to track. A wild guess says
that jRuby’s has probably got the best coverage, but is there anyone
actually knee-deep in them that can give me a better idea?

Thanks,

2

On Jul 31, 10:35 am, Alex Y. [email protected] wrote:

Has anyone got an overview of the current status of the various Ruby
spec suites? I know of at least 3 (rubicon, jRuby’s tests, and
rubinius’ specs), but I’ve got no idea how complete or active they are,
or which Ruby version they’re attempting to track. A wild guess says
that jRuby’s has probably got the best coverage, but is there anyone
actually knee-deep in them that can give me a better idea?

You need to differentiate between JRuby’s own tests and the ones they
are importing. They also include Rubinius and Rubicon ones in their
repository.

I’m working together with Rubinius to make their spec better. I’d say
that it is the most complete spec for the classes we have already
completed.

Here’s some numbers that are easy to extract: (I’d really like someone
to do a more complete comparison on this, though.)

Lines of code:

freya:~/svn/rubinius/trunk flgr$ wc -l spec/core/*.rb | sort:
[…]
1000 spec/core/file_spec.rb
1331 spec/core/hash_spec.rb
3473 spec/core/array_spec.rb
5540 spec/core/string_spec.rb
17831 total

Note that I’d consider hash and array quite complete. You can have a
look for yourself at
http://code.fallingsnow.net/svn/rubinius/trunk/spec/core/

Number of assertions: (this could be off a bit)

freya:~/svn/rubinius/trunk flgr$ grep -R should spec/core/ | grep -v
‘#’ | egrep -v specify|it | wc -l
11058

Number of tests:

freya:~/svn/rubinius/trunk flgr$ bin/mspec -t r spec/core 2> /dev/null
| grep examples
2243 examples, 38 failures

According to the out-dated comparison from Christian N.
(http://chneukirchen.org/blog/archive/2007/05/analyzing-the-jruby-test-
suite.html) this would make us the biggest test suite.

We’ve more than doubled from 8.4 kLOC to 17.8 kLOC in two months.

Oh, and I totally ignored the non-core specs. There also is language
and library specs, but those might be less complete than the core
ones.

And we’re always looking for new contributors. Specifying Ruby
completely is a huge task, but one that contributes a lot to the
community by making it possible to find bugs in implementations that
will cause compatibility issues if they go overlooked. If we want to
have one single Ruby language in the future instead of five different
and incompatible flavours we will need this.

3

M. Edward (Ed) Borasky wrote:

be the most complete. It’s the successor to Rubicon. As I recall, there
are two “Ruby standards” – MRI (Matz’ Reference Implementation), which
is Ruby 1.8.x and which jRuby, MRI itself, IronRuby, and probably most
other implementations are attempting to track – and KRI (Koichi’s
Reference Implementation), which is Ruby 1.9.x – Ruby 2.

Pat Eyler is the person who most closely tracks the various
implementations of Ruby, so he’d be the logical one to go into more detail.

Fab. Thanks, guys.

4

Alex Y. wrote:

Has anyone got an overview of the current status of the various Ruby
spec suites? I know of at least 3 (rubicon, jRuby’s tests, and
rubinius’ specs), but I’ve got no idea how complete or active they are,
or which Ruby version they’re attempting to track. A wild guess says
that jRuby’s has probably got the best coverage, but is there anyone
actually knee-deep in them that can give me a better idea?

Thanks,
There is a gem, the Big “Formal” Test Suite (bfts) that is supposed to
be the most complete. It’s the successor to Rubicon. As I recall, there
are two “Ruby standards” – MRI (Matz’ Reference Implementation), which
is Ruby 1.8.x and which jRuby, MRI itself, IronRuby, and probably most
other implementations are attempting to track – and KRI (Koichi’s
Reference Implementation), which is Ruby 1.9.x – Ruby 2.

Pat Eyler is the person who most closely tracks the various
implementations of Ruby, so he’d be the logical one to go into more
detail.

5

M. Edward (Ed) Borasky wrote:

be the most complete. It’s the successor to Rubicon. As I recall, there
are two “Ruby standards” – MRI (Matz’ Reference Implementation), which
is Ruby 1.8.x and which jRuby, MRI itself, IronRuby, and probably most
other implementations are attempting to track – and KRI (Koichi’s
Reference Implementation), which is Ruby 1.9.x – Ruby 2.

Pat Eyler is the person who most closely tracks the various
implementations of Ruby, so he’d be the logical one to go into more detail.

Fab. Thanks, guys.

6

M. Edward (Ed) Borasky wrote:

be the most complete. It’s the successor to Rubicon. As I recall, there
are two “Ruby standards” – MRI (Matz’ Reference Implementation), which
is Ruby 1.8.x and which jRuby, MRI itself, IronRuby, and probably most
other implementations are attempting to track – and KRI (Koichi’s
Reference Implementation), which is Ruby 1.9.x – Ruby 2.

Pat Eyler is the person who most closely tracks the various
implementations of Ruby, so he’d be the logical one to go into more detail.

Fab. Thanks, guys.

7

Alex Y. wrote:

Fab. Thanks, guys.

Bah. Sorry for the noise…