I re-implemented parts of the Rubinius test suite harness to use
features that are implemented in IronRuby. The good news is that we’re
using the specs as written without any changes at all.
The spec suite driver currently looks at Array, Hash and String only.
If I run the driver under MRI, I see 17 failures out of 743 specs; only
one failure is expected. The other 16 failures are due the fact that I
couldn’t implement that part of the test suite harness without using
features that IronRuby doesn’t have today. So that’s not a bad baseline
If I run the same spec suite under IronRuby, we see 457 failures out of
743, for a pass rate of 38%. If we omit String, which is the least
implemented of these types, our pass rate for Array and Hash rises to
58% (149 fail out of 353).
Some of the failures are obviously due to features that aren’t
implemented at all (mostly in String). Some of them are due to a class
of language features that aren’t implemented (e.g. taint). Some are due
to known DLR bugs like the one that Martin is currently consumed with
fixing. The remainder are real bugs in our implementation.
The spec harness is a fairly sophisticated Ruby program. The team should
be proud of getting the language to this state so quickly!
I’m attaching the test dump to this mail for the curious. The shelveset
that contains this stuff should be ready to review tomorrow and will be
pushed out to SVN right after that.