We’ve been having a lot of problems around here with getting RubySpecs
integrated properly, so again my apologies for delays in getting the
latest sources out to y’all.
We have an old version of Ruby checked into SNAP (1.8.6 patch level 0),
and we’re having problems updating it. The current RubySpecs will crash
MRI when we try to get some baseline numbers. We also had a bug in
IronRuby that caused it to return incorrect exit codes. This meant that
regression tests were broken in SNAP for some time. These are fixed now,
and it’s a matter of getting our specs to pass SNAP in their current
form.
I manually drove the specs tonight to get a feel for where we are at.
These are the raw numbers for the core RubySpecs:
Ruby 1.8.6 (September 24 2007 build, patch level 111)
1092 files, 4753 examples, 16793 expectations, 101 failures, 328 errors
[43 types]
Ruby 1.8.6 (March 13 2007 build patch level 0):
995 files, 3924 examples, 10755 expectations, 101 failures, 312 errors
[41 types] **
IronRuby (build from this weekend)
936 files, 3384 examples, 12026 expectations, 440 failures, 450 errors
[41 types] *
- IronRuby crashes on the thread and threadgroup specs right now
** low number because String specs cause segfault and string has 68
files, 798 examples, 5948 expectations with 6 failures under 1.8.6 patch
level 111
Percentages relative to 1.8.6 patch level 111:
85.7% of spec files run
71.2% of examples pass
71.6% of expectations pass
A bit of bad news though- we have a memory leak in IronRuby today (we’re
not invalidating the DLR cache elements correctly, so a cache without a
scavenging policy is a memory leak :)). This is just killing our
performance for large suites like the RubySpecs, which essentially run
all of those 16793 expectations in a single process. We actually run out
of memory today in this test under 32 bit Windows(!) We know how to fix
the memory leak, but it’s going to take some time to do as it’s queued
up behind some prep work that we need to do first.
Thanks,
-John