Forum: Ruby-core 1.9.2 Should Pass RubySpec Before Release

I humbly suggest that a prerequisite of 1.9.2 being released is that
it passes RubySpec.

1.9 HEAD currently has over 350 failures and 638 errors with RubySpec.
It is likely that many of these are caused by intentional changes
between 1.8 and 1.9, but it is equally likely that some represent
bugs. Until these tests pass we simply won't know.

RubySpec is the best tool we have to understand the implications of
changes in the language and its libraries. Test failures not only help
discover bugs, but force us to clarify how we expect Ruby to work, and
its components to interact.

I appreciate that everybody on this list is busy with their own
projects, so I intend to do the majority of this work myself, with the
assistance of the other RubySpec contributors. I believe that I can
fix the failing 'core' and 'language' tests before Christmas. It is
unlikely that I'll be able to fix all of the 'library' tests as well.

To achieve this goal I need to ask a lot of questions, but wish to do
so in a way that neither floods ruby-core nor angers the core
committers. So I ask: how can I work best with the core team and
members of this list? I suspect that the best approach will be for me
to post my questions in this thread. That enables disinterested
parties to easily filter me out, while signposting questions for the
core team. I could even post bug reports here before filing them on
the tracker, if that would be easier. IOW, how can I annoy you the
least? :-)

The other thing that would help is an explanation of any general
principles of the 1.8 -> 1.9 changes. For example, one is that, in
general, methods that modify a String should preserve its encoding.
That's useful, because I can apply it to many test cases.

(If by some remote chance anybody reading this doesn't know about
RubySpec, see http://rubyspec.org/ , stop by #rubyspec, or e-mail me
off-list).

> 1.9 HEAD currently has over 350 failures and 638 errors with RubySpec.
> It is likely that many of these are caused by intentional changes
> between 1.8 and 1.9, but it is equally likely that some represent
> bugs. Until these tests pass we simply won't know.

Not to mention

# Bug: 187 open on 755
on trunk

That could use some love.

Perhaps you could post the various failures somewhere and core people
could look at it to determine which are bugs?  Are any of the failures
related to the outstanding open bugs?


Perhaps we could have a few saturdays dedicated as official bug scrubs
for trunk where everyone unites to try to fix things.

Oh I was going to mention re: 1.9.2 that ruby-prof does work on 1.9.1
if you do an SVN co of its code, just so you're aware [there is a high
speed profiler that works, I think].
Thoughts?
=r

> Perhaps you could post the various failures somewhere and core people
> could look at it to determine which are bugs?

If there's any interest, I'll gladly do that. My impression is that
they'd far prefer potential bugs reduced to trivial test cases and
filed as separate tickets.

> Are any of the failures related to the outstanding open bugs?

I would suspect they are. Likely the minority, however.

> Perhaps we could have a few saturdays dedicated as official bug scrubs
> for trunk where everyone unites to try to fix things.

Heh, I'll believe that when I see it. ;-)

> Oh I was going to mention re: 1.9.2 that ruby-prof does work on 1.9.1
> if you do an SVN co of its code, just so you're aware [there is a high
> speed profiler that works, I think].

Thanks. I don't forsee much need for a profiler. The difficult part is
establishing the general cause of a test failure then determining
whether said cause equates to intentional change of functionality or
an accidental defect. That's why I wanted discussion as to the best
way to pose my questions.

To start with, please could I have answers to the following threads?

* http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/23706
* http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/23699

Thanks!

An update on the remaining 'core' spec failures under 1.9 HEAD:

1) Array#hash properly handles recursive arrays ERROR
ArgumentError: recursive key for hash

2) Array#& properly handles recursive arrays ERROR
ArgumentError: recursive key for hash

3) Array#- properly handles recursive arrays ERROR
ArgumentError: recursive key for hash

4) Array#| properly handles recursive arrays ERROR
ArgumentError: recursive key for hash

5) Array#uniq properly handles recursive arrays ERROR
ArgumentError: recursive key for hash

6) Array#uniq! properly handles recursive arrays ERROR
ArgumentError: recursive key for hash

These are all dependent on the resolution of ticket #1852
(Enumerable's #hash Raises ArgumentError When Recursive Values are
Present) <http://redmine.ruby-lang.org/issues/show/1852>

7) Fiber#resume executes the ensure clause FAILED
Expected :begin to equal :ensure

This depends on ticket #595 (Fiber ignores ensure clause)
<http://redmine.ruby-lang.org/issues/show/595>

8) Hash#hash generates a hash for recursive hash structures ERROR
ArgumentError: recursive key for hash

Also dependent on ticket #1852 (Enumerable's #hash Raises
ArgumentError When Recursive Values are Present)
<http://redmine.ruby-lang.org/issues/show/1852>.

9) IO#binmode does not raise any errors on closed stream FAILED
Expected to not get Exception

This was reported as ticket #2046 ((IO|File)#binmode is Inconsistent
Across Versions) <http://redmine.ruby-lang.org/issues/show/2046>

10) IO#reopen reassociates self with a new stream after some reads 
FAILED
Expected nil to equal "Line 1: One\n"

Queried in ticket #1755 (IO#reopen Doesn't Fully Associate with Given
Stream on 1.9; Ignores pos on 1.8)
<http://redmine.ruby-lang.org/issues/show/1755>.

11) Kernel#freeze has no effect on immediate values FAILED
Expected true to equal false

Reported as ticket #1747 (Immediate Values Can be Frozen)
<http://redmine.ruby-lang.org/issues/show/1747>

12) String#upto calls block with self even if self is less than stop
but stop length is less than self length FAILED
Expected [] to equal ["25"]

Probably a version-specific difference, but queried in ticket #2049
(String#upto: Possible Regression)
<http://redmine.ruby-lang.org/issues/show/2049>

13) Thread#raise on a sleeping thread re-raises active exception FAILED
Expected ZeroDivisionError but got RuntimeError ()

14) Thread#raise on a running thread re-raises active exception FAILED
Expected ZeroDivisionError but got RuntimeError ()

These are the subject of ticket #2048 (Thread#raise: Handling of
Current Exception) <http://redmine.ruby-lang.org/issues/show/2048>.

15) Time#<=> returns nil when Time is compared to some Object ERROR
NoMethodError: undefined method `<=>' for #<Object:0x11f52c90>

This was reported as ticket #2047 (Time#<=> Raises NoMethodError on
Incomparable Argument) <http://redmine.ruby-lang.org/issues/show/2047>

Finished in 51.214144 seconds

1409 files, 7949 examples, 96543 expectations, 7 failures, 8 errors

An update on the remaining 'core' spec failures under 1.9 HEAD:

1) Array#hash properly handles recursive arrays ERROR
ArgumentError: recursive key for hash

2) Array#& properly handles recursive arrays ERROR
ArgumentError: recursive key for hash

3) Array#- properly handles recursive arrays ERROR
ArgumentError: recursive key for hash

4) Array#| properly handles recursive arrays ERROR
ArgumentError: recursive key for hash

5) Array#uniq properly handles recursive arrays ERROR
ArgumentError: recursive key for hash

6) Array#uniq! properly handles recursive arrays ERROR
ArgumentError: recursive key for hash

These are all dependent on the resolution of ticket #1852
(Enumerable's #hash Raises ArgumentError When Recursive Values are
Present) <http://redmine.ruby-lang.org/issues/show/1852>

7) Fiber#resume executes the ensure clause FAILED
Expected :begin to equal :ensure

This depends on ticket #595 (Fiber ignores ensure clause)
<http://redmine.ruby-lang.org/issues/show/595>

8) Hash#hash generates a hash for recursive hash structures ERROR
ArgumentError: recursive key for hash

Also dependent on ticket #1852 (Enumerable's #hash Raises
ArgumentError When Recursive Values are Present)
<http://redmine.ruby-lang.org/issues/show/1852>.

9) IO#binmode does not raise any errors on closed stream FAILED
Expected to not get Exception

This was reported as ticket #2046 ((IO|File)#binmode is Inconsistent
Across Versions) <http://redmine.ruby-lang.org/issues/show/2046>

10) IO#reopen reassociates self with a new stream after some reads 
FAILED
Expected nil to equal "Line 1: One\n"

Queried in ticket #1755 (IO#reopen Doesn't Fully Associate with Given
Stream on 1.9; Ignores pos on 1.8)
<http://redmine.ruby-lang.org/issues/show/1755>.

11) Kernel#freeze has no effect on immediate values FAILED
Expected true to equal false

Reported as ticket #1747 (Immediate Values Can be Frozen)
<http://redmine.ruby-lang.org/issues/show/1747>

12) String#upto calls block with self even if self is less than stop
but stop length is less than self length FAILED
Expected [] to equal ["25"]

Probably a version-specific difference, but queried in ticket #2049
(String#upto: Possible Regression)
<http://redmine.ruby-lang.org/issues/show/2049>

13) Thread#raise on a sleeping thread re-raises active exception FAILED
Expected ZeroDivisionError but got RuntimeError ()

14) Thread#raise on a running thread re-raises active exception FAILED
Expected ZeroDivisionError but got RuntimeError ()

These are the subject of ticket #2048 (Thread#raise: Handling of
Current Exception) <http://redmine.ruby-lang.org/issues/show/2048>.

15) Time#<=> returns nil when Time is compared to some Object ERROR
NoMethodError: undefined method `<=>' for #<Object:0x11f52c90>

This was reported as ticket #2047 (Time#<=> Raises NoMethodError on
Incomparable Argument) <http://redmine.ruby-lang.org/issues/show/2047>

Finished in 51.214144 seconds

1409 files, 7949 examples, 96543 expectations, 7 failures, 8 errors