Forum: IronRuby Some feedback on the compile/test process

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
91eb330fb36d1e03c856574dfb77d2bc?d=identicon&s=25 Thibaut Barrère (thbar)
on 2009-04-15 15:19
(Received via mailing list)
Some notes as I go though the contributing process, in hope that will
be useful to make it smoother:
- after a clean checkout and launching dev.bat, brbd would not compile
(today). I believe it reported a missing microsoft.scripting.dll
issue.
- opening the solution in VS2008 shows 14 warnings about referenced
components that cannot be found (System, mscorlib, System.Net), not
sure if that's an issue or not
- compiling through VS2008 makes the compile using brbd works
afterwards.
- first pass at launching irtests opens a bunch of windows - most of
them fail with failure to load default.mspec (see [1]) - after
searching, the message was caused by an old .mspecrc that was sitting
in my home folder. Removing it solved the issue.
- I noticed a runfirst.cmd, should I actually run it or is it obsolete
? It seems to have an error in it (references default.mspec.rb which
has been renamed to default.mspec since).
- on a french machine, the MiscTests/NumericLiterals1 fails see [2] -
sounds like a missing InvariantCulture in a float ToString call. I'll
look into it unless someone fixes it in between.

On irtests execution: the end of irtests launches irtest.bat, run.bat
and 4 mspec in parallels ("start" calls). Is there a need for that (ie
one process that monitors the others) or could it be run sequentially
if it's more comfortable for me ? The load is putting my machine down
on its knees.

cheers,

-- Thibaut

===== [1] mspec error =====

IronRuby 0.3 0.3.0.0 on .NET 2.0.0.0
:0:in `load': no such file to load --
C:\git\ironruby\Merlin\Main\..\External\Languages\IronRuby\mspec\default.mspec
(LoadError)

===== [2] NumericLiterals1 failure =====

1) NumericLiterals1
c:\git\ironruby\Merlin\Main\Languages\Ruby\IronRuby.Tests\Runtime\MiscTests.cs
: 411
Unexpected output:

'1\r\n-1\r\n1\r\n1,1\r\n-1,1\r\n1,1\r\n2.0\r\n2.0\r\n2,156\r\n'.

First difference (8):
actual = ','
expected = '.'

Repro: C:\git\ironruby\Merlin\Main\bin\debug\IronRuby.Tests.exe
NumericLiterals1
E1deaac6561498a0e7472e6268ea9ccc?d=identicon&s=25 Jimmy Schementi (Guest)
on 2009-04-15 16:14
(Received via mailing list)
> (today). I believe it reported a missing microsoft.scripting.dll
> issue.

Please remove the ClrAssembly.csproj file from the solution ... then all
will build fine. That will be removed shortly.

> - opening the solution in VS2008 shows 14 warnings about referenced
> components that cannot be found (System, mscorlib, System.Net), not
> sure if that's an issue or not

Hmm, weird. The build configuration was definitely Debug, not
"Silverlight Debug" or something like that?

> - compiling through VS2008 makes the compile using brbd works
> afterwards.
> - first pass at launching irtests opens a bunch of windows - most of
> them fail with failure to load default.mspec (see [1]) - after
> searching, the message was caused by an old .mspecrc that was sitting
> in my home folder. Removing it solved the issue.

If you run dev.bat first, this problem should go away. Irtest opens a
bunch of windows, one for each test runner, simply for parallelism.

> - I noticed a runfirst.cmd, should I actually run it or is it obsolete
> ? It seems to have an error in it (references default.mspec.rb which
> has been renamed to default.mspec since).

Runfirst.cmd should be removed, dev.bat is what you want

> - on a french machine, the MiscTests/NumericLiterals1 fails see [2] -
> sounds like a missing InvariantCulture in a float ToString call. I'll
> look into it unless someone fixes it in between.

Please track this with a bug on codeplex, someone else might have
context.

> On irtests execution: the end of irtests launches irtest.bat, run.bat
> and 4 mspec in parallels ("start" calls). Is there a need for that (ie
> one process that monitors the others) or could it be run sequentially
> if it's more comfortable for me ? The load is putting my machine down
> on its knees.

You could make a version of irtest that does them sequentially
91eb330fb36d1e03c856574dfb77d2bc?d=identicon&s=25 Thibaut Barrère (thbar)
on 2009-04-15 16:25
(Received via mailing list)
A couple of steps later: I meet the following failures while running
irtests on an unmodified fork (by window title, see matching commands
at the end of the mail):
- smoke tests => 1 failure
- library rubyspec tests => 4 failures, 15 errors in various places
- core rubyspec tests => 1 failure, 1 error

I suspect some of these are linked to the fact that I use a french
culture on my machine.

Just do get a point of comparison and know what needs fixing, are all
these supposed to pass, or are some known to fail and that would be
normal ?

cheers,

-- Thibaut

C:\Git\ironruby\Merlin\Main\Languages\Ruby>start "Smoke Tests"
C:\git\ironruby\Merlin\Main\Languages\Ruby\Tests\Scripts\irtest.bat
C:\Git\ironruby\Merlin\Main\Languages\Ruby>start "Legacy Tests"
C:\git\ironruby\Merlin\Main\Languages\Ruby\Tests\run.bat
C:\Git\ironruby\Merlin\Main\Languages\Ruby>start "Core RubySpec tests"
mspec ci -fd -V :core
C:\Git\ironruby\Merlin\Main\Languages\Ruby>start "Language RubySpec
tests" mspec ci -fd -V :lang
C:\Git\ironruby\Merlin\Main\Languages\Ruby>start "Library RubySpec
tests" mspec ci -fd -V :lib
C:\Git\ironruby\Merlin\Main\Languages\Ruby>start "Command Line
RubySpec tests" mspec ci -fd -V :cli :netinterop
E1deaac6561498a0e7472e6268ea9ccc?d=identicon&s=25 Jimmy Schementi (Guest)
on 2009-04-15 17:05
(Received via mailing list)
Yes, these are all suppose to pass ... they run "mspec ci" which is the
continuous integration mode ... which omits "critical" or "fails" tagged
tests.

It'd be good to know where IronRuby falls over in different cultures, so
feel free to report bugs on codeplex about these things.
91eb330fb36d1e03c856574dfb77d2bc?d=identicon&s=25 Thibaut Barrère (thbar)
on 2009-04-16 10:01
(Received via mailing list)
> Yes, these are all suppose to pass ... they run "mspec ci" which is the continuous 
integration mode ... which omits "critical" or "fails" tagged tests.
>
> It'd be good to know where IronRuby falls over in different cultures, so feel free to 
report bugs on codeplex about these things.

thanks - I added the first one here:
http://ironruby.codeplex.com/WorkItem/View.aspx?Wo...

and will add or fix more as I go.

-- Thibaut
91eb330fb36d1e03c856574dfb77d2bc?d=identicon&s=25 Thibaut Barrère (thbar)
on 2009-04-16 10:01
(Received via mailing list)
Hi Jimmy,

thanks for the feedback - comments inlined below:

> Please remove the ClrAssembly.csproj file from the solution ... then all will build 
fine. That will be removed shortly.

thanks.

>> - opening the solution in VS2008 shows 14 warnings about referenced
>> components that cannot be found (System, mscorlib, System.Net), not
>> sure if that's an issue or not
>
> Hmm, weird. The build configuration was definitely Debug, not "Silverlight Debug" or 
something like that?

It is Debug, not Silverlight Debug. I looked at the .csproj and saw
conditional references to these dll when SilverlightDebug is set.
It seems that the condition is not evaluated properly. Not a big deal
though, it seems to work, but could lead to weird things I guess.

>> - I noticed a runfirst.cmd, should I actually run it or is it obsolete
>> ? It seems to have an error in it (references default.mspec.rb which
>> has been renamed to default.mspec since).
>
> Runfirst.cmd should be removed, dev.bat is what you want

ok, thanks.

>
>> - on a french machine, the MiscTests/NumericLiterals1 fails see [2] -
>> sounds like a missing InvariantCulture in a float ToString call. I'll
>> look into it unless someone fixes it in between.
>
> Please track this with a bug on codeplex, someone else might have context.

Done: http://ironruby.codeplex.com/WorkItem/View.aspx?Wo...

>> On irtests execution: the end of irtests launches irtest.bat, run.bat
>> and 4 mspec in parallels ("start" calls). Is there a need for that (ie
>> one process that monitors the others) or could it be run sequentially
>> if it's more comfortable for me ? The load is putting my machine down
>> on its knees.
>
> You could make a version of irtest that does them sequentially

Thanks - much faster.

-- Thibaut
1c29f9b1bf5f1b88ed8b0c9a9be39788?d=identicon&s=25 Daniele Alessandri (Guest)
on 2009-04-16 12:06
(Received via mailing list)
I confirm that I can reproduce this bug on my system too (it-IT). See
below for a simpler test case:

>>> puts 1.1
1,1
=> nil
>>> p 1.1
1.1
=> nil

The problem is in IoOps.ToPrintedString, where there is indeed an
obj.ToString() for doubles. Converting it with Convert.ToString(obj,
System.Globalization.CultureInfo.InvariantCulture) seems to fix the
issue, but I have just tried it in the REPL without running the test
suite yet. If you want, I can commit this fix in my repository if it
passes the tests.

On Thu, Apr 16, 2009 at 09:58, Thibaut Barrère
<thibaut.barrere@gmail.com> wrote:
> _______________________________________________
> Ironruby-core mailing list
> Ironruby-core@rubyforge.org
> http://rubyforge.org/mailman/listinfo/ironruby-core
>


--
Daniele Alessandri
http://www.clorophilla.net/blog/
http://twitter.com/JoL1hAHN
91eb330fb36d1e03c856574dfb77d2bc?d=identicon&s=25 Thibaut Barrère (thbar)
on 2009-04-16 12:18
(Received via mailing list)
Haya,

> The problem is in IoOps.ToPrintedString, where there is indeed an
> obj.ToString() for doubles. Converting it with Convert.ToString(obj,
> System.Globalization.CultureInfo.InvariantCulture) seems to fix the
> issue, but I have just tried it in the REPL without running the test
> suite yet. If you want, I can commit this fix in my repository if it
> passes the tests.

puts 1.1 gives 1,1 here too.

Please commit the fix, this way I'll be able to see if this solves
other failing tests in french too (I suppose it will).

I'll have to figure out how to cherry pick one commit from your
branch, the git way :)

-- Thibaut
91eb330fb36d1e03c856574dfb77d2bc?d=identicon&s=25 Thibaut Barrère (thbar)
on 2009-04-16 12:41
(Received via mailing list)
Attachment: remaining-errors.txt (2 KB)
Attachment: ioops.diff (1 KB)
Hi,

>> The problem is in IoOps.ToPrintedString, where there is indeed an
>> obj.ToString() for doubles. Converting it with Convert.ToString(obj,
>> System.Globalization.CultureInfo.InvariantCulture) seems to fix the
>> issue, but I have just tried it in the REPL without running the test
>> suite yet. If you want, I can commit this fix in my repository if it
>> passes the tests.

I ran the whole irtests suite after applying the attached patch, based
on what Daniel described.

The good news is that the smoke tests now pass on my french machine.
Library / core errors remain unchanged (I included a log of the
remaining core errors in case someone has more time for this today).

cheers,

-- Thibaut
806ef55917f436850d2c4b0a5703e7cc?d=identicon&s=25 Jirapong Nanta (jirapong)
on 2009-04-16 13:40
(Received via mailing list)
I got 14 warning as well, My workaround was to change a Target
Framework to .NET 3.5

Thanks,
-Jirapong
1c29f9b1bf5f1b88ed8b0c9a9be39788?d=identicon&s=25 Daniele Alessandri (Guest)
on 2009-04-16 17:48
(Received via mailing list)
Attachment: IoOps.ToPrintedString.diff (2 KB)
On Thu, Apr 16, 2009 at 12:35, Thibaut Barrère
<thibaut.barrere@gmail.com> wrote:

> I ran the whole irtests suite after applying the attached patch, based
> on what Daniel described.

You can find a slightly modified patch attached to this mail, this is
what I'm going to push on my repository. I think this one is better
than the first one as it removes the overhead of Convert.ToString and
it reduces the number of unboxing operations from object to double.
Aea6cfe04952626ab630bde47ff82f89?d=identicon&s=25 Shri Borde (Guest)
on 2009-04-16 19:56
(Received via mailing list)
Thibaut, I have a change to irtests.bat which will give an option to run
the tests in series or in parallel. I will push this through in my next
commit so folks can chose how they want to run the tests...

The one issue with running in series is that if a task fails, you want
to run the remaining tasks as well so that you can have the full list of
failures. However, if you just keep running all tasks in series, the
output of the failing task can scroll out, and then you lose the
information of which test failed. The right solution is to redirect the
output of all the tasks so that you can inspect the log files to see the
details of the failure.

Setting up the output redirection, etc, is more work. A rake task would
be nice so that all the logic can be written in Ruby rather than in
irtests.bat. I will leave this for another day. Let's see how far
irtests.bat can scale...

Thanks,
Shri
91eb330fb36d1e03c856574dfb77d2bc?d=identicon&s=25 Thibaut Barrère (thbar)
on 2009-04-17 15:15
(Received via mailing list)
Hi Shri,

> Thibaut, I have a change to irtests.bat which will give an option to run the tests in 
series or in parallel. I will push this through in my next commit so folks can chose how 
they want to run the tests...
>
> The one issue with running in series is that if a task fails, you want to run the 
remaining tasks as well so that you can have the full list of failures. However, if you 
just keep running all tasks in series, the output of the failing task can scroll out, and 
then you lose the information of which test failed. The right solution is to redirect the 
output of all the tasks so that you can inspect the log files to see the details of the 
failure.
>
> Setting up the output redirection, etc, is more work. A rake task would be nice so that 
all the logic can be written in Ruby rather than in irtests.bat. I will leave this for 
another day. Let's see how far irtests.bat can scale...

thanks for the change! And yeah, let's see how the batch can scale. If
it gets boring, I'll try to take a stab at redirecting output.

cheers

-- Thibaut
This topic is locked and can not be replied to.