# Ruby test suites? **URL:** \ **Category:** Ruby\ **Created:** [July 31, 2007, 10:37am UTC](https://www.ruby-forum.com/t/ruby-test-suites/110820 "2007-07-31T10:37:02Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![regularfry](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/45deac/32.png) [@regularfry](https://www.ruby-forum.com/u/regularfry)\ **Post date:** [July 31, 2007, 10:37am UTC](https://www.ruby-forum.com/t/ruby-test-suites/110820/1 "2007-07-31T10:37:02Z")
Has anyone got an overview of the current status of the various Ruby spec suites? I know of at least 3 (rubicon, jRuby’s tests, and rubinius’ specs), but I’ve got no idea how complete or active they are, or which Ruby version they’re attempting to track. A wild guess says that jRuby’s has probably got the best coverage, but is there anyone actually knee-deep in them that can give me a better idea? Thanks, ---
**Author:** ![regularfry](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/45deac/32.png) [@regularfry](https://www.ruby-forum.com/u/regularfry)\ **Post date:** [July 31, 2007, 1:16pm UTC](https://www.ruby-forum.com/t/ruby-test-suites/110820/2 "2007-07-31T13:16:24Z")
On Jul 31, 10:35 am, Alex Y. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > Has anyone got an overview of the current status of the various Ruby > spec suites? I know of at least 3 (rubicon, jRuby’s tests, and > rubinius’ specs), but I’ve got no idea how complete or active they are, > or which Ruby version they’re attempting to track. A wild guess says > that jRuby’s has probably got the best coverage, but is there anyone > actually knee-deep in them that can give me a better idea? You need to differentiate between JRuby’s own tests and the ones they are importing. They also include Rubinius and Rubicon ones in their repository. I’m working together with Rubinius to make their spec better. I’d say that it is the most complete spec for the classes we have already completed. Here’s some numbers that are easy to extract: (I’d really like someone to do a more complete comparison on this, though.) Lines of code: freya:~/svn/rubinius/trunk flgr$ wc -l spec/core/\*.rb | sort: […] 1000 spec/core/file\_spec.rb 1331 spec/core/hash\_spec.rb 3473 spec/core/array\_spec.rb 5540 spec/core/string\_spec.rb 17831 total Note that I’d consider hash and array quite complete. You can have a look for yourself at [http://code.fallingsnow.net/svn/rubinius/trunk/spec/core/](http://code.fallingsnow.net/svn/rubinius/trunk/spec/core/) Number of assertions: (this could be off a bit) freya:~/svn/rubinius/trunk flgr$ grep -R should spec/core/ | grep -v ‘#’ | egrep -v specify|it | wc -l 11058 Number of tests: freya:~/svn/rubinius/trunk flgr$ bin/mspec -t r spec/core 2\> /dev/null | grep examples 2243 examples, 38 failures According to the out-dated comparison from Christian N. ([http://chneukirchen.org/blog/archive/2007/05/analyzing-the-jruby-test-](http://chneukirchen.org/blog/archive/2007/05/analyzing-the-jruby-test-) suite.html) this would make us the biggest test suite. We’ve more than doubled from 8.4 kLOC to 17.8 kLOC in two months. Oh, and I totally ignored the non-core specs. There also is language and library specs, but those might be less complete than the core ones. And we’re always looking for new contributors. Specifying Ruby completely is a huge task, but one that contributes a lot to the community by making it possible to find bugs in implementations that will cause compatibility issues if they go overlooked. If we want to have one single Ruby language in the future instead of five different and incompatible flavours we will need this. ---
**Author:** ![regularfry](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/45deac/32.png) [@regularfry](https://www.ruby-forum.com/u/regularfry)\ **Post date:** [July 31, 2007, 5:40pm UTC](https://www.ruby-forum.com/t/ruby-test-suites/110820/3 "2007-07-31T17:40:06Z")
M. Edward (Ed) Borasky wrote: > be the most complete. It’s the successor to Rubicon. As I recall, there > are two “Ruby standards” – MRI (Matz’ Reference Implementation), which > is Ruby 1.8.x and which jRuby, MRI itself, IronRuby, and probably most > other implementations are attempting to track – and KRI (Koichi’s > Reference Implementation), which is Ruby 1.9.x – Ruby 2. > > Pat Eyler is the person who most closely tracks the various > implementations of Ruby, so he’d be the logical one to go into more detail. Fab. Thanks, guys. ---
**Author:** ![regularfry](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/45deac/32.png) [@regularfry](https://www.ruby-forum.com/u/regularfry)\ **Post date:** [July 31, 2007, 4:50pm UTC](https://www.ruby-forum.com/t/ruby-test-suites/110820/4 "2007-07-31T16:50:56Z")
Alex Y. wrote: > Has anyone got an overview of the current status of the various Ruby > spec suites? I know of at least 3 (rubicon, jRuby’s tests, and > rubinius’ specs), but I’ve got no idea how complete or active they are, > or which Ruby version they’re attempting to track. A wild guess says > that jRuby’s has probably got the best coverage, but is there anyone > actually knee-deep in them that can give me a better idea? > > Thanks, > There is a gem, the Big “Formal” Test Suite (bfts) that is supposed to > be the most complete. It’s the successor to Rubicon. As I recall, there > are two “Ruby standards” – MRI (Matz’ Reference Implementation), which > is Ruby 1.8.x and which jRuby, MRI itself, IronRuby, and probably most > other implementations are attempting to track – and KRI (Koichi’s > Reference Implementation), which is Ruby 1.9.x – Ruby 2. Pat Eyler is the person who most closely tracks the various implementations of Ruby, so he’d be the logical one to go into more detail. ---
**Author:** ![regularfry](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/45deac/32.png) [@regularfry](https://www.ruby-forum.com/u/regularfry)\ **Post date:** [July 31, 2007, 5:42pm UTC](https://www.ruby-forum.com/t/ruby-test-suites/110820/5 "2007-07-31T17:42:06Z")
M. Edward (Ed) Borasky wrote: > be the most complete. It’s the successor to Rubicon. As I recall, there > are two “Ruby standards” – MRI (Matz’ Reference Implementation), which > is Ruby 1.8.x and which jRuby, MRI itself, IronRuby, and probably most > other implementations are attempting to track – and KRI (Koichi’s > Reference Implementation), which is Ruby 1.9.x – Ruby 2. > > Pat Eyler is the person who most closely tracks the various > implementations of Ruby, so he’d be the logical one to go into more detail. Fab. Thanks, guys. ---
**Author:** ![regularfry](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/45deac/32.png) [@regularfry](https://www.ruby-forum.com/u/regularfry)\ **Post date:** [July 31, 2007, 5:44pm UTC](https://www.ruby-forum.com/t/ruby-test-suites/110820/6 "2007-07-31T17:44:06Z")
M. Edward (Ed) Borasky wrote: > be the most complete. It’s the successor to Rubicon. As I recall, there > are two “Ruby standards” – MRI (Matz’ Reference Implementation), which > is Ruby 1.8.x and which jRuby, MRI itself, IronRuby, and probably most > other implementations are attempting to track – and KRI (Koichi’s > Reference Implementation), which is Ruby 1.9.x – Ruby 2. > > Pat Eyler is the person who most closely tracks the various > implementations of Ruby, so he’d be the logical one to go into more detail. Fab. Thanks, guys. ---
**Author:** ![regularfry](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/45deac/32.png) [@regularfry](https://www.ruby-forum.com/u/regularfry)\ **Post date:** [July 31, 2007, 5:52pm UTC](https://www.ruby-forum.com/t/ruby-test-suites/110820/7 "2007-07-31T17:52:08Z")
Alex Y. wrote: > Fab. Thanks, guys. Bah. Sorry for the noise…