Forum: IronRuby RubySpecs

D7aaf307f96bbdf5ff39be313fe581be?d=identicon&s=25 Orion Edwards (Guest)
on 2011-10-31 21:32
(Received via mailing list)
I've just run mspec under the Languages\Ruby\Tests\mspec\rubyspec\core
folder against MRI 1.9.2p0 (which is checked into the ironlanguages-main
git repo), and it crashes with a segfault in putc_spec.rb

I then tried running against MRI 1.9.2p290 and I get the following:

       1518 files, 9618 examples, 28229 expectations, 339 failures, 256
errors

Running against the dev build of IronRuby stalls running tests in the
array subfolder, so I can't even complete the run.

I'd like to try fixing some of the spec failures that IronRuby has, but
these results raise several questions:

1. Are the specs in the mspec\rubyspec folder up to date? How would
someone find this out and/or update them?
  - Also, I added a few specs the other day for the marshalling code. Do
these need to somehow get pushed upstream into some "master" rubyspec
repository??

2. Shouldn't we check the latest build of MRI into the ironlangauges
git-repo? Either the latest 1.9.2 or the just-released 1.9.3 instead?

3. Isn't MRI the "definitive" ruby? How can the specs be failing against
MRI?

4. If I update some IronRuby code to pass the specs, how do I know that
the specs are even correct?

Thanks, Orion
Cb51033949ffccd982ae32c9f890f25a?d=identicon&s=25 Tomas Matousek (Guest)
on 2011-10-31 22:16
(Received via mailing list)
1) Partly. We are in progress of updating them. Since we've added more
specs and sometimes corrected existing ones this needs to be done with
care. The current progress is captured here:

https://gist.github.com/1159998

After we are done with this process of bringing the specs up to date we
can submit a path to RubySpec containing the changed we made. The plan
is also to make the RubySpec dir a submodule so that merging with
RubySpec is easier.

2) Or maybe we should just remove MRI completely? You can always use one
on your system. I need to check if we need CRuby for anything in the
infrastructure.

3) Well, that would be a question for RubySpec maintainers. I found some
specs be failures Windows specific, which means the specs are not
properly written. Feel free to contribute to RubySpecs git repo - I bet
they are happy to accept fixes.

4) Well, you need to figure out if the behavior makes sense. If it
doesn't feel free to file a bug on CRuby and let them decide if the
behavior is intentional or not. Then CRuby might get fixed, specs might
get fixed, or IronRuby might get fixed.


Which specs particularly are you working on? Is it marshal related?

Tomas

From: ironruby-core-bounces@rubyforge.org
[mailto:ironruby-core-bounces@rubyforge.org] On Behalf Of Orion Edwards
Sent: Monday, October 31, 2011 1:19 PM
To: ironruby-core@rubyforge.org
Subject: [Ironruby-core] RubySpecs

I've just run mspec under the Languages\Ruby\Tests\mspec\rubyspec\core
folder against MRI 1.9.2p0 (which is checked into the ironlanguages-main
git repo), and it crashes with a segfault in putc_spec.rb

I then tried running against MRI 1.9.2p290 and I get the following:

       1518 files, 9618 examples, 28229 expectations, 339 failures, 256
errors

Running against the dev build of IronRuby stalls running tests in the
array subfolder, so I can't even complete the run.

I'd like to try fixing some of the spec failures that IronRuby has, but
these results raise several questions:

1. Are the specs in the mspec\rubyspec folder up to date? How would
someone find this out and/or update them?
  - Also, I added a few specs the other day for the marshalling code. Do
these need to somehow get pushed upstream into some "master" rubyspec
repository??

2. Shouldn't we check the latest build of MRI into the ironlangauges
git-repo? Either the latest 1.9.2 or the just-released 1.9.3 instead?

3. Isn't MRI the "definitive" ruby? How can the specs be failing against
MRI?

4. If I update some IronRuby code to pass the specs, how do I know that
the specs are even correct?

Thanks, Orion
D7aaf307f96bbdf5ff39be313fe581be?d=identicon&s=25 Orion Edwards (Guest)
on 2011-11-01 00:43
(Received via mailing list)
Thanks for that.

After the code review comments about Exception#== I was running the
Exception specs. 3 fail against ruby 1.9.2p290 (2011-07-09)
[i386-mingw32]

I don't have access to a linux/mac system at work to check if they pass
on
non-windows platforms, but it doesn't look like there's anything
platform
related in those errors, so I'd doubt it.

Random comment:
It appears that there's a lot more work to be done in IronRuby
particularly relating to Errno and SystemCallErrors - I could do this
work, but there's a blocking problem:
 - Various bits of IronRuby core code call helper methods such as
RubyExceptions.CreateEINVAL, RubyExceptions.CreateEEXIST, etc, etc.
These
are supposed to return Errno::EINVAL, but they can't, because the Errno
classes are defined in IronRuby.Libraries and the
RubyExceptions.CreateXYZ
methods are defined in IronRuby.dll. We'd have to move the Errno stuff
out
of Libraries and into IronRuby.dll I think



From:   Tomas Matousek <Tomas.Matousek@microsoft.com>
To:     "ironruby-core@rubyforge.org" <ironruby-core@rubyforge.org>
Date:   01/11/2011 10:16 a.m.
Subject:        Re: [Ironruby-core] RubySpecs
Sent by:        ironruby-core-bounces@rubyforge.org



1) Partly. We are in progress of updating them. Since we?ve added more
specs and sometimes corrected existing ones this needs to be done with
care. The current progress is captured here:

https://gist.github.com/1159998

After we are done with this process of bringing the specs up to date we
can submit a path to RubySpec containing the changed we made. The plan
is
also to make the RubySpec dir a submodule so that merging with RubySpec
is
easier.

2) Or maybe we should just remove MRI completely? You can always use one
on your system. I need to check if we need CRuby for anything in the
infrastructure.

3) Well, that would be a question for RubySpec maintainers. I found some
specs be failures Windows specific, which means the specs are not
properly
written. Feel free to contribute to RubySpecs git repo ? I bet they are
happy to accept fixes.

4) Well, you need to figure out if the behavior makes sense. If it
doesn?t
feel free to file a bug on CRuby and let them decide if the behavior is
intentional or not. Then CRuby might get fixed, specs might get fixed,
or
IronRuby might get fixed.


Which specs particularly are you working on? Is it marshal related?

Tomas

From: ironruby-core-bounces@rubyforge.org [
mailto:ironruby-core-bounces@rubyforge.org] On Behalf Of Orion Edwards
Sent: Monday, October 31, 2011 1:19 PM
To: ironruby-core@rubyforge.org
Subject: [Ironruby-core] RubySpecs

I've just run mspec under the Languages\Ruby\Tests\mspec\rubyspec\core
folder against MRI 1.9.2p0 (which is checked into the ironlanguages-main
git repo), and it crashes with a segfault in putc_spec.rb

I then tried running against MRI 1.9.2p290 and I get the following:

       1518 files, 9618 examples, 28229 expectations, 339 failures, 256
errors

Running against the dev build of IronRuby stalls running tests in the
array subfolder, so I can't even complete the run.

I'd like to try fixing some of the spec failures that IronRuby has, but
these results raise several questions:

1. Are the specs in the mspec\rubyspec folder up to date? How would
someone find this out and/or update them?
  - Also, I added a few specs the other day for the marshalling code. Do
these need to somehow get pushed upstream into some "master" rubyspec
repository??

2. Shouldn't we check the latest build of MRI into the ironlangauges
git-repo? Either the latest 1.9.2 or the just-released 1.9.3 instead?

3. Isn't MRI the "definitive" ruby? How can the specs be failing against
MRI?

4. If I update some IronRuby code to pass the specs, how do I know that
the specs are even correct?

Thanks, Orion_______________________________________________
Ironruby-core mailing list
Ironruby-core@rubyforge.org
http://rubyforge.org/mailman/listinfo/ironruby-core
Cb51033949ffccd982ae32c9f890f25a?d=identicon&s=25 Tomas Matousek (Guest)
on 2011-11-01 01:12
(Received via mailing list)
There is a lot of work to do around exceptions in general. I wouldn't
start digging into it if I were you :). I'm thinking of some core
changes in how we map Ruby to .NET exceptions... it will take me some
time to get to it though. I would not worry about all these Errno
exceptions for now unless your application is blocked by some specific
bug. Certainly not if a few specs fail here and there (we have thousands
of specs failing, which might be more important to fix).

Time to file a bug on CRuby to get some input from MRI guys what they
think the behavior should really be if it doesn't make sense. Or
RubySpec guys if the MRI behavior is reasonable.

Tomas

From: ironruby-core-bounces@rubyforge.org
[mailto:ironruby-core-bounces@rubyforge.org] On Behalf Of Orion Edwards
Sent: Monday, October 31, 2011 4:40 PM
To: ironruby-core@rubyforge.org
Subject: Re: [Ironruby-core] RubySpecs

Thanks for that.

After the code review comments about Exception#== I was running the
Exception specs. 3 fail against ruby 1.9.2p290 (2011-07-09)
[i386-mingw32]

I don't have access to a linux/mac system at work to check if they pass
on non-windows platforms, but it doesn't look like there's anything
platform related in those errors, so I'd doubt it.

Random comment:
It appears that there's a lot more work to be done in IronRuby
particularly relating to Errno and SystemCallErrors - I could do this
work, but there's a blocking problem:
 - Various bits of IronRuby core code call helper methods such as
RubyExceptions.CreateEINVAL, RubyExceptions.CreateEEXIST, etc, etc.
These are supposed to return Errno::EINVAL, but they can't, because the
Errno classes are defined in IronRuby.Libraries and the
RubyExceptions.CreateXYZ methods are defined in IronRuby.dll. We'd have
to move the Errno stuff out of Libraries and into IronRuby.dll I think



From:        Tomas Matousek
<Tomas.Matousek@microsoft.com<mailto:Tomas.Matousek@microsoft.com>>
To:
"ironruby-core@rubyforge.org<mailto:ironruby-core@rubyforge.org>"
<ironruby-core@rubyforge.org<mailto:ironruby-core@rubyforge.org>>
Date:        01/11/2011 10:16 a.m.
Subject:        Re: [Ironruby-core] RubySpecs
Sent by:
ironruby-core-bounces@rubyforge.org<mailto:ironruby-core-bounces@rubyforge.org>
________________________________



1) Partly. We are in progress of updating them. Since we've added more
specs and sometimes corrected existing ones this needs to be done with
care. The current progress is captured here:

https://gist.github.com/1159998

After we are done with this process of bringing the specs up to date we
can submit a path to RubySpec containing the changed we made. The plan
is also to make the RubySpec dir a submodule so that merging with
RubySpec is easier.

2) Or maybe we should just remove MRI completely? You can always use one
on your system. I need to check if we need CRuby for anything in the
infrastructure.

3) Well, that would be a question for RubySpec maintainers. I found some
specs be failures Windows specific, which means the specs are not
properly written. Feel free to contribute to RubySpecs git repo - I bet
they are happy to accept fixes.

4) Well, you need to figure out if the behavior makes sense. If it
doesn't feel free to file a bug on CRuby and let them decide if the
behavior is intentional or not. Then CRuby might get fixed, specs might
get fixed, or IronRuby might get fixed.


Which specs particularly are you working on? Is it marshal related?

Tomas

From:
ironruby-core-bounces@rubyforge.org<mailto:ironruby-core-bounces@rubyforge.org>
[mailto:ironruby-core-bounces@rubyforge.org] On Behalf Of Orion Edwards
Sent: Monday, October 31, 2011 1:19 PM
To: ironruby-core@rubyforge.org<mailto:ironruby-core@rubyforge.org>
Subject: [Ironruby-core] RubySpecs

I've just run mspec under the Languages\Ruby\Tests\mspec\rubyspec\core
folder against MRI 1.9.2p0 (which is checked into the ironlanguages-main
git repo), and it crashes with a segfault in putc_spec.rb

I then tried running against MRI 1.9.2p290 and I get the following:

      1518 files, 9618 examples, 28229 expectations, 339 failures, 256
errors

Running against the dev build of IronRuby stalls running tests in the
array subfolder, so I can't even complete the run.

I'd like to try fixing some of the spec failures that IronRuby has, but
these results raise several questions:

1. Are the specs in the mspec\rubyspec folder up to date? How would
someone find this out and/or update them?
 - Also, I added a few specs the other day for the marshalling code. Do
these need to somehow get pushed upstream into some "master" rubyspec
repository??

2. Shouldn't we check the latest build of MRI into the ironlangauges
git-repo? Either the latest 1.9.2 or the just-released 1.9.3 instead?

3. Isn't MRI the "definitive" ruby? How can the specs be failing against
MRI?

4. If I update some IronRuby code to pass the specs, how do I know that
the specs are even correct?

Thanks, Orion_______________________________________________
Ironruby-core mailing list
Ironruby-core@rubyforge.org<mailto:Ironruby-core@rubyforge.org>
http://rubyforge.org/mailman/listinfo/ironruby-core
This topic is locked and can not be replied to.