Forum: IronRuby RubySpecs

I've just run mspec under the Languages\Ruby\Tests\mspec\rubyspec\core
folder against MRI 1.9.2p0 (which is checked into the ironlanguages-main
git repo), and it crashes with a segfault in putc_spec.rb

I then tried running against MRI 1.9.2p290 and I get the following:

       1518 files, 9618 examples, 28229 expectations, 339 failures, 256
errors

Running against the dev build of IronRuby stalls running tests in the
array subfolder, so I can't even complete the run.

I'd like to try fixing some of the spec failures that IronRuby has, but
these results raise several questions:

1. Are the specs in the mspec\rubyspec folder up to date? How would
someone find this out and/or update them?
  - Also, I added a few specs the other day for the marshalling code. Do
these need to somehow get pushed upstream into some "master" rubyspec
repository??

2. Shouldn't we check the latest build of MRI into the ironlangauges
git-repo? Either the latest 1.9.2 or the just-released 1.9.3 instead?

3. Isn't MRI the "definitive" ruby? How can the specs be failing against
MRI?

4. If I update some IronRuby code to pass the specs, how do I know that
the specs are even correct?

Thanks, Orion

1) Partly. We are in progress of updating them. Since we've added more 
specs and sometimes corrected existing ones this needs to be done with 
care. The current progress is captured here:

https://gist.github.com/1159998

After we are done with this process of bringing the specs up to date we 
can submit a path to RubySpec containing the changed we made. The plan 
is also to make the RubySpec dir a submodule so that merging with 
RubySpec is easier.

2) Or maybe we should just remove MRI completely? You can always use one 
on your system. I need to check if we need CRuby for anything in the 
infrastructure.

3) Well, that would be a question for RubySpec maintainers. I found some 
specs be failures Windows specific, which means the specs are not 
properly written. Feel free to contribute to RubySpecs git repo - I bet 
they are happy to accept fixes.

4) Well, you need to figure out if the behavior makes sense. If it 
doesn't feel free to file a bug on CRuby and let them decide if the 
behavior is intentional or not. Then CRuby might get fixed, specs might 
get fixed, or IronRuby might get fixed.


Which specs particularly are you working on? Is it marshal related?

Tomas

From: ironruby-core-bounces@rubyforge.org 
[mailto:ironruby-core-bounces@rubyforge.org] On Behalf Of Orion Edwards
Sent: Monday, October 31, 2011 1:19 PM
To: ironruby-core@rubyforge.org
Subject: [Ironruby-core] RubySpecs

I've just run mspec under the Languages\Ruby\Tests\mspec\rubyspec\core 
folder against MRI 1.9.2p0 (which is checked into the ironlanguages-main 
git repo), and it crashes with a segfault in putc_spec.rb

I then tried running against MRI 1.9.2p290 and I get the following:

       1518 files, 9618 examples, 28229 expectations, 339 failures, 256 
errors

Running against the dev build of IronRuby stalls running tests in the 
array subfolder, so I can't even complete the run.

I'd like to try fixing some of the spec failures that IronRuby has, but 
these results raise several questions:

1. Are the specs in the mspec\rubyspec folder up to date? How would 
someone find this out and/or update them?
  - Also, I added a few specs the other day for the marshalling code. Do 
these need to somehow get pushed upstream into some "master" rubyspec 
repository??

2. Shouldn't we check the latest build of MRI into the ironlangauges 
git-repo? Either the latest 1.9.2 or the just-released 1.9.3 instead?

3. Isn't MRI the "definitive" ruby? How can the specs be failing against 
MRI?

4. If I update some IronRuby code to pass the specs, how do I know that 
the specs are even correct?

Thanks, Orion

Thanks for that.

After the code review comments about Exception#== I was running the
Exception specs. 3 fail against ruby 1.9.2p290 (2011-07-09) 
[i386-mingw32]

I don't have access to a linux/mac system at work to check if they pass 
on
non-windows platforms, but it doesn't look like there's anything 
platform
related in those errors, so I'd doubt it.

Random comment:
It appears that there's a lot more work to be done in IronRuby
particularly relating to Errno and SystemCallErrors - I could do this
work, but there's a blocking problem:
 - Various bits of IronRuby core code call helper methods such as
RubyExceptions.CreateEINVAL, RubyExceptions.CreateEEXIST, etc, etc. 
These
are supposed to return Errno::EINVAL, but they can't, because the Errno
classes are defined in IronRuby.Libraries and the 
RubyExceptions.CreateXYZ
methods are defined in IronRuby.dll. We'd have to move the Errno stuff 
out
of Libraries and into IronRuby.dll I think



From:   Tomas Matousek <Tomas.Matousek@microsoft.com>
To:     "ironruby-core@rubyforge.org" <ironruby-core@rubyforge.org>
Date:   01/11/2011 10:16 a.m.
Subject:        Re: [Ironruby-core] RubySpecs
Sent by:        ironruby-core-bounces@rubyforge.org



1) Partly. We are in progress of updating them. Since we?ve added more
specs and sometimes corrected existing ones this needs to be done with
care. The current progress is captured here:

https://gist.github.com/1159998

After we are done with this process of bringing the specs up to date we
can submit a path to RubySpec containing the changed we made. The plan 
is
also to make the RubySpec dir a submodule so that merging with RubySpec 
is
easier.

2) Or maybe we should just remove MRI completely? You can always use one
on your system. I need to check if we need CRuby for anything in the
infrastructure.

3) Well, that would be a question for RubySpec maintainers. I found some
specs be failures Windows specific, which means the specs are not 
properly
written. Feel free to contribute to RubySpecs git repo ? I bet they are
happy to accept fixes.

4) Well, you need to figure out if the behavior makes sense. If it 
doesn?t
feel free to file a bug on CRuby and let them decide if the behavior is
intentional or not. Then CRuby might get fixed, specs might get fixed, 
or
IronRuby might get fixed.


Which specs particularly are you working on? Is it marshal related?

Tomas

From: ironruby-core-bounces@rubyforge.org [
mailto:ironruby-core-bounces@rubyforge.org] On Behalf Of Orion Edwards
Sent: Monday, October 31, 2011 1:19 PM
To: ironruby-core@rubyforge.org
Subject: [Ironruby-core] RubySpecs

I've just run mspec under the Languages\Ruby\Tests\mspec\rubyspec\core
folder against MRI 1.9.2p0 (which is checked into the ironlanguages-main
git repo), and it crashes with a segfault in putc_spec.rb

I then tried running against MRI 1.9.2p290 and I get the following:

       1518 files, 9618 examples, 28229 expectations, 339 failures, 256
errors

Running against the dev build of IronRuby stalls running tests in the
array subfolder, so I can't even complete the run.

I'd like to try fixing some of the spec failures that IronRuby has, but
these results raise several questions:

1. Are the specs in the mspec\rubyspec folder up to date? How would
someone find this out and/or update them?
  - Also, I added a few specs the other day for the marshalling code. Do
these need to somehow get pushed upstream into some "master" rubyspec
repository??

2. Shouldn't we check the latest build of MRI into the ironlangauges
git-repo? Either the latest 1.9.2 or the just-released 1.9.3 instead?

3. Isn't MRI the "definitive" ruby? How can the specs be failing against
MRI?

4. If I update some IronRuby code to pass the specs, how do I know that
the specs are even correct?

Thanks, Orion_______________________________________________
Ironruby-core mailing list
Ironruby-core@rubyforge.org
http://rubyforge.org/mailman/listinfo/ironruby-core

There is a lot of work to do around exceptions in general. I wouldn't 
start digging into it if I were you :). I'm thinking of some core 
changes in how we map Ruby to .NET exceptions... it will take me some 
time to get to it though. I would not worry about all these Errno 
exceptions for now unless your application is blocked by some specific 
bug. Certainly not if a few specs fail here and there (we have thousands 
of specs failing, which might be more important to fix).

Time to file a bug on CRuby to get some input from MRI guys what they 
think the behavior should really be if it doesn't make sense. Or 
RubySpec guys if the MRI behavior is reasonable.

Tomas

From: ironruby-core-bounces@rubyforge.org 
[mailto:ironruby-core-bounces@rubyforge.org] On Behalf Of Orion Edwards
Sent: Monday, October 31, 2011 4:40 PM
To: ironruby-core@rubyforge.org
Subject: Re: [Ironruby-core] RubySpecs

Thanks for that.

After the code review comments about Exception#== I was running the 
Exception specs. 3 fail against ruby 1.9.2p290 (2011-07-09) 
[i386-mingw32]

I don't have access to a linux/mac system at work to check if they pass 
on non-windows platforms, but it doesn't look like there's anything 
platform related in those errors, so I'd doubt it.

Random comment:
It appears that there's a lot more work to be done in IronRuby 
particularly relating to Errno and SystemCallErrors - I could do this 
work, but there's a blocking problem:
 - Various bits of IronRuby core code call helper methods such as 
RubyExceptions.CreateEINVAL, RubyExceptions.CreateEEXIST, etc, etc. 
These are supposed to return Errno::EINVAL, but they can't, because the 
Errno classes are defined in IronRuby.Libraries and the 
RubyExceptions.CreateXYZ methods are defined in IronRuby.dll. We'd have 
to move the Errno stuff out of Libraries and into IronRuby.dll I think



From:        Tomas Matousek 
<Tomas.Matousek@microsoft.com<mailto:Tomas.Matousek@microsoft.com>>
To: 
"ironruby-core@rubyforge.org<mailto:ironruby-core@rubyforge.org>" 
<ironruby-core@rubyforge.org<mailto:ironruby-core@rubyforge.org>>
Date:        01/11/2011 10:16 a.m.
Subject:        Re: [Ironruby-core] RubySpecs
Sent by: 
ironruby-core-bounces@rubyforge.org<mailto:ironruby-core-bounces@rubyforge.org>
________________________________



1) Partly. We are in progress of updating them. Since we've added more 
specs and sometimes corrected existing ones this needs to be done with 
care. The current progress is captured here:

https://gist.github.com/1159998

After we are done with this process of bringing the specs up to date we 
can submit a path to RubySpec containing the changed we made. The plan 
is also to make the RubySpec dir a submodule so that merging with 
RubySpec is easier.

2) Or maybe we should just remove MRI completely? You can always use one 
on your system. I need to check if we need CRuby for anything in the 
infrastructure.

3) Well, that would be a question for RubySpec maintainers. I found some 
specs be failures Windows specific, which means the specs are not 
properly written. Feel free to contribute to RubySpecs git repo - I bet 
they are happy to accept fixes.

4) Well, you need to figure out if the behavior makes sense. If it 
doesn't feel free to file a bug on CRuby and let them decide if the 
behavior is intentional or not. Then CRuby might get fixed, specs might 
get fixed, or IronRuby might get fixed.


Which specs particularly are you working on? Is it marshal related?

Tomas

From: 
ironruby-core-bounces@rubyforge.org<mailto:ironruby-core-bounces@rubyforge.org> 
[mailto:ironruby-core-bounces@rubyforge.org] On Behalf Of Orion Edwards
Sent: Monday, October 31, 2011 1:19 PM
To: ironruby-core@rubyforge.org<mailto:ironruby-core@rubyforge.org>
Subject: [Ironruby-core] RubySpecs

I've just run mspec under the Languages\Ruby\Tests\mspec\rubyspec\core 
folder against MRI 1.9.2p0 (which is checked into the ironlanguages-main 
git repo), and it crashes with a segfault in putc_spec.rb

I then tried running against MRI 1.9.2p290 and I get the following:

      1518 files, 9618 examples, 28229 expectations, 339 failures, 256 
errors

Running against the dev build of IronRuby stalls running tests in the 
array subfolder, so I can't even complete the run.

I'd like to try fixing some of the spec failures that IronRuby has, but 
these results raise several questions:

1. Are the specs in the mspec\rubyspec folder up to date? How would 
someone find this out and/or update them?
 - Also, I added a few specs the other day for the marshalling code. Do 
these need to somehow get pushed upstream into some "master" rubyspec 
repository??

2. Shouldn't we check the latest build of MRI into the ironlangauges 
git-repo? Either the latest 1.9.2 or the just-released 1.9.3 instead?

3. Isn't MRI the "definitive" ruby? How can the specs be failing against 
MRI?

4. If I update some IronRuby code to pass the specs, how do I know that 
the specs are even correct?

Thanks, Orion_______________________________________________
Ironruby-core mailing list
Ironruby-core@rubyforge.org<mailto:Ironruby-core@rubyforge.org>
http://rubyforge.org/mailman/listinfo/ironruby-core