Regarding IronRuby... How true it sounds from this blog

Ruby IronRuby
21

Peter Bacon D. wrote:

I have not personally been affected by the long time periods between the
code drops. I doubt it has had a serious impact on many other contributors.
Although there have certainly been a few issues highlighted in the mailing
list around the hosting API and some other internal chunks of code that have
delayed people trying to work on top of IR - the SaphireSteel guys come to
mind, I still believe this has not really been a problem, since even in a
totally open environment, people would need to wait on the core team to
deliver API’s anyway, no?

In general the problem I see most often goes like this:

  1. Contributor tries to run IronRuby from RubyForge trunk.
  2. Contributor finds a bug.
  3. Contributor emails the list, asking if this bug has been fixed, and
    often volunteering to fix it.
  4. IronRuby Team Member replies, saying the bug is fixed and will be in
    the next drop.
  5. Contributor waits, maybe moving on to other IronRuby work or maybe
    walking away from the project until the next drop.

And this seems to happen again and again. Not only does it slow the
process of fixing bugs, it makes it impossible for people to want to
help fix them. If you can never know you’re running against the latest
sources, the process of finding a bug, emailing to see if it’s fixed
already, and probably waiting for that fix to arrive is extremely
discouraging.

  • Charlie
22

2008/4/29 John L. (IRONRUBY) [email protected]:

Has working on the SVN sources (with the attendant delays in propagating to / from our version of ‘the truth’) blocked you from working on a contribution?

Blocked? No. Discouraged? Hell sure.

(The same applies to IronPython, but IronPython is much more stable
than IronRuby, so it changes less frequently, which makes keeping up
to date easier.)

23

On Mon, 28 Apr 2008 10:05:43 -0600, Charles Oliver N.
[email protected] wrote:

And this seems to happen again and again. Not only does it slow the
process of fixing bugs, it makes it impossible for people to want to
help fix them. If you can never know you’re running against the latest
sources, the process of finding a bug, emailing to see if it’s fixed
already, and probably waiting for that fix to arrive is extremely
discouraging.

I agree. If I am remembering correctly, one of the primary reasons
behind
the dual-repository approach is the need to run a myriad of internal
tests
from a test suite that reaches farther and deeper than just IronRuby,
and
therefore can not see the light of day outside the MSFT firewall.

John, is this an accurate assessment? If yes, while I certainly
recognize
the need to run the code against internal test suites, couldn’t it be
looked at from the opposite perspective?That of: We, the community, tell
you, the big bad corporate firewall, when you get to gain access to
our
code to run your tests. We will continue on our way checking it
whatever
we want whenever we want, and you can use repository revisions as a
marker
to determine what can be viewed as “blessed” and what can not as far as
releases are concerned.

If it really is an issue with intellectual property et. al, you can keep
those results locked up in a bit locker that guarantees they’ll never
experience life outside their darkened dungeon. We, the community, are
not
interested in the results of internal tests, and we certainly would
understand that, regardless of the results of our external tests, there
are certain check boxes that need to be checked by powers unknown to us
before an officially blessed release can be made. All we care about is
passing the spec, something which is, quite obviously, controlled
outside
the grasp of Redmond’s barbed [fire]wire-trimmed walls. If it takes a
few
extra weeks to take a particular revision of the repository through the
internal ringer before getting the official rubber stamp, then so be it.
It wouldn’t be getting in the way of development progress, and if not
mistaken, this is really the core of the argument as to why the process
is
currently broken.

Food for thought…

–
/M:D

M. David P.
Co-Founder & Chief Architect, 3rd&Urban, LLC
Email: [email protected] | [email protected]
Mobile: (206) 999-0588
http://3rdandUrban.com | http://amp.fm |
M. David Peterson

24

“but since removed all signs of the Ruby.NET
parser/scanner in favor of a from-the-ground-up implementation written
entirely by – I believe – Tomas M.”

More precisely, I’ve heavily refactored the tokenizer (and it’s still
not finished) and rewrote semantic actions in the grammar to create
IronRuby AST - which I wrote from scratch. The grammar rules themselves
are more or less as they was in Ruby.NET. With some renames and minor
changes.

Tomas

25

On Mon, 28 Apr 2008 10:03:49 -0600, Sanghyeon S. [email protected]
wrote:

Blocked? No. Discouraged? Hell sure.

I can’t think of a better way to accurately portray the reality of the
situation. Nicely stated, Seo!

–
/M:D

M. David P.
Co-Founder & Chief Architect, 3rd&Urban, LLC
Email: [email protected] | [email protected]
Mobile: (206) 999-0588
http://3rdandUrban.com | http://amp.fm |
M. David Peterson

26

I can’t say that the delay has stopped me from working on a contribution
since I’ve just recently started investigating where I may be able to
lend a
hand. However, I can say that I’m much more reluctant to jump in and
start
working on something since I don’t want to work on a implementing or
fixing
something that has already been addressed.

I think it would be great to have a more open environment where the
community knows what the members of the IronRuby team are working on and
what they’re currently thinking about, as well as what things they
aren’t
able to get to yet but are higher priority. This would help the
community
understand what they should stay away from, as well as what would be a
good
place to contribute. In order to foster a community around IronRuby I
believe there needs to be something that can rally the community around
a
cause. Right now, most of the people I’ve spoken to are in a wait and
see
mode with IronRuby. It would be great if we could do something to get
people in a “what can I do to contribute value to the project” mode.

This is coming from someone who has just recently “joined” the IronRuby
community, so I may not be looking in the right places.

Cheers,
Steve

On Mon, Apr 28, 2008 at 11:35 AM, John L. (IRONRUBY)
[email protected]

27

On Mon, 28 Apr 2008 10:42:01 -0600, Tomas M.
[email protected] wrote:

More precisely, I’ve heavily refactored the tokenizer (and it’s still
not finished) and rewrote semantic actions in the grammar to create
IronRuby AST - which I wrote from scratch. The grammar rules themselves
are more or less as they was in Ruby.NET. With some renames and minor
changes.

Thanks for the clarification, Tomas!

–
/M:D

M. David P.
Co-Founder & Chief Architect, 3rd&Urban, LLC
Email: [email protected] | [email protected]
Mobile: (206) 999-0588
http://3rdandUrban.com | http://amp.fm |
M. David Peterson

28

So, one of the “details” that are wrong is that we don’t support
multiple isolated engines in a single process.
It’s actually quite simple to do so via DLR Hosting API. The main
concept here is ScriptRuntime. This class represents the world for a
dynamic language. The class holds on loaded assemblies, .NET namespace
references, etc. Each language could also associate its own global state
with the runtime. IronRuby has all Ruby global state there: global
variables table, class hierarchy etc. So, unless you use CLR interop,
there is no way how the script could get outside this sandbox. If you
want to isolate the runtimes even more (for CLR interop) you can always
create a ScriptRuntime inside a separate app-domain.

Let’s show some example (a self-contained C# source code of a simple
IronRuby host follows):

using System.IO;
using Microsoft.Scripting.Hosting;

class RubyHostingExample {
    public static void Main() {
        const string write = @"C:\Temp\write.rb";
        const string read = @"C:\Temp\read.rb";

        File.WriteAllText(write, @"

$x = ‘Hello from runtime #1!’
C = ‘some constant’

module Kernel
def say_bye
puts ‘bye’
end
end
");

        File.WriteAllText(read, @"

puts $x
if defined? C
puts C
else
puts ‘C not defined’
end
say_bye rescue puts $!
puts
");

        ScriptRuntime runtime1 = ScriptRuntime.Create();
        ScriptRuntime runtime2 = ScriptRuntime.Create();

        runtime1.ExecuteFile(write);
        runtime2.ExecuteFile(read);
        runtime1.ExecuteFile(read);
    }

}

Let’s compile and run it:

C:\IronRuby\Bin\Debug>csc /r:Microsoft.Scripting.dll
/r:Microsoft.Scripting.Core.dll /r:IronRuby.dll rt.cs
Microsoft (R) Visual C# 2008 Compiler version 3.5.21022.8
for Microsoft (R) .NET Framework version 3.5
Copyright (C) Microsoft Corporation. All rights reserved.

C:\IronRuby\Bin\Debug>rt.exe
nil
C not defined
undefined local variable or method `say_bye’ for main:Object

Hello from runtime #1!
some constant
bye

And if you want app-domain isolation, just do
ScriptRuntime.Create(System.AppDomain.CreateDomain(“foo”)).

That was easy, wasn’t it?

Tomas

29

Tomas M. wrote:

[snip…]

And if you want app-domain isolation, just do ScriptRuntime.Create(System.AppDomain.CreateDomain(“foo”)).

Does this actually work? No one has been able to post working code on
creating an IronPython engine in another app domain on the IronPython
mailing list.

Can you use this to place restrictions on the app domain - like restrict
which assemblies can be loaded and control network / filesystem access?

A working example would be a wonderful thing…

The reason I am dubious is that it seems that the code generation used
by the DLR requires pretty much full trust in .NET 2, so any
restrictions (which is usually the point of running in another app
domain) blow up. I would dearly love to be proved wrong on this of
course.

Michael F.

30

M. David P. wrote:

parser/scanner in favor of a from-the-ground-up implementation written
entirely by – I believe – Tomas M… Of course, as Charlie points
out somewhat correctly in his opening paragraph,

The IronRuby parser/scanner being bootstrapped by the Ruby.NET
parser/scanner is certainly enough to say that’s where IronRuby’s roots
lie. And even without that, IronRuby probably wouldn’t have been
attempted if Ruby.NET had shown it to be too difficult or impossible.
IronRuby owes Ruby.NET for its birth, at least.

IronRuby was still Wilco Bauer’s IronRuby, a doomed codebase and
project name eventually to be adopted by Microsoft’s later Ruby
implementation effort.

… which is at least partially correct, if not a bit misleading given
that for all intents and purposes the IronRuby project of today is a
from-the-ground-up implementation of the Ruby language and runtime based
on top of the from-the-ground-up Dynamic Language Runtime code base and
architecture.

No, it’s an entirely correct statement. At the time, IronRuby was
Wilco’s project, and no IronRuby work had been started at MS. I did not
make any claim that the codebase was somehow reused or incorporated into
the official “IronRuby”, and made a point of calling it doomed because
as far as I know it’s never going to be touched again.

Perhaps I should have said:

“IronRuby” was still Wilco Bauer’s IronRuby, …

  • Charlie
31

Here’s a working example (no partial trust) in Python:

import clr
clr.AddReference(‘Microsoft.Scripting.Core’)

import System
from Microsoft.Scripting import SourceCodeKind
from Microsoft.Scripting.Hosting import ScriptRuntime

ad = System.AppDomain.CreateDomain(‘foo’)
sr = ScriptRuntime.Create(ad)
sr.LoadAssembly(clr.GetClrType(str).Assembly)
py = sr.GetEngine(‘py’)
su = py.CreateScriptSourceFromString(‘import System\nprint
System.AppDomain.CurrentDomain\n’, SourceCodeKind.File)
su.Execute()

Indeed partial trust might be a problem - we’ve run into a few bugs
there on the desktop CLR where we have divergent code paths from
Silverlight. It’s also only available on Orcas / .NET 2.0 Sp1 or later
where we’ll use anonymously hosted dynamic methods if they’re available.
Finally I believe our “optimized module” and other subclassing of .NET
standard types won’t work because those need full reflection.emit but I
haven’t verified that.

Anyway, the issue of partial trust has been brought up with various
people on the DLR side of things and there should be a push at some
point to ensure this works.

But there are other advantages w/ app domains than just security. You
also get:
the ability to unload code w/ a decreased worry of corruption
This is using Thread.Abort safely. Large amounts of
code can be safely unloaded because there’s no shared state outside of
the app domain which can be corrupted (note there’s some code in the
world that this doesn’t apply to, but it applies to most of the .NET
framework).
isolation of static variables that live outside of the script
environment
I think Tomas alluded to this
the ability to unload assemblies
Ye-olde-reason to use app domains on the CLR

32

In a technical fashion? No. From an emotional standpoint? Yes.
Right now IronRuby is very unstable from the view of an outside
contributor, you don’t know if the code you’re working on now is going
to need /major/ changes in the next drop, and you don’t know when
that’s going to be. Why work on a bug that “in truth” may already be
fixed?

The most important change that MSFT can do is let you push to
rubyforge DIRECTLY, none of this internal updates pushed to rubyforge
once in a while. I assume it’s corporate preventing this, because it
really make no sense otherwise. What we have here isn’t an OSS
community project in the traditional sense, what we have is a
Microsoft project that they’ve so kindly, in their infinite wisdom
allow us commoners to work on now and then. Oh but you can’t see or
touch the real code until we’re ready to let you. This is HIGHLY
discouraging.

Don’t get me wrong, I applaud Microsoft for going this far, it’s a
major step, but only a step, there’s still a long way to go.

On Mon, Apr 28, 2008 at 11:35 AM, John L. (IRONRUBY)
[email protected] wrote:

–
Michael L.
[Polymath Prokrammer]
http://blog.prokrams.com

33

I believe one of the key problems is the DLR. As I understand, it MS
makes
a distinction between “important” stuff (i.e. the DLR) and “peripheral”
stuff (i.e. IronXxxx). MS wants to have complete control over the DLR
and
is not interested in making it Open Source. Rather the DLR code is just
community viewable, much like the rest of the .NET framework code. I
can
understand this since core .NET Framework code is central to the MS
strategy
and they don’t want things sneaking in the sides.

IronRuby, IronPython and so on are not so important to MS strategy and
they
are more happy to let the community muck about with the code. I believe
that the long term goal is to open up the IronXxxx code much more to the
community but the problem is that the line between the DLR and the
IronXxxx
languages is not yet nailed down. Therefore until that happens MS is
unlikely to hand over the project to the community.

I would be interested to know how often an SVN dump is created compared
to
successful check-ins going through the SNAP process. Ideally, every
successful SNAP check-in should get automatically dumped out on the
RubyForge SVN repos, whether it added value or broke the tests or
whatever.
You can always have SVN tags on the “good” builds and also create
downloadable “good” releases on the RubyForge site - this point would
probably help Justin B.'s access problems too.

Another scenario, which /M:D alludes to if I understand correctly, is to
allow the community to modify the code in the RubyForge project and then
let
MS select “good” builds to check back into the Team system via the SNAP
process. That way, the community feels ownership of the project and MS
get
that quality control on what finally goes into IronRuby. There are
obviously many technical hurdles to overcome before this could become
reality. In particular, there needs to be a separation of DLR from
IronXxxx, including, probably, some kind of stable release of the DLR
for
the IronXxxx projects to work off.

Any other ideas? John what are you thinking here?

Pete

34

On Mon, Apr 28, 2008 at 4:19 PM, Michael F.
[email protected] wrote:

Tomas M. wrote:

[snip…]

And if you want app-domain isolation, just do
ScriptRuntime.Create(System.AppDomain.CreateDomain(“foo”)).

Does this actually work? No one has been able to post working code on
creating an IronPython engine in another app domain on the IronPython
mailing list.

Actually, I seem to recall that this works fine in IronPython –
provided that you’re running under FullTrust. (Which, as you pointed
out, needs to be addressed.)

–
Curt H.
[email protected]

35

Dino Viehland wrote:

Here’s a working example (no partial trust) in Python:

Cool! Thanks Dino. I don’t normally speak to you here. :slight_smile:

Michael

36

2008/4/29 Peter Bacon D. [email protected]:

I believe one of the key problems is the DLR. As I understand, it MS makes
a distinction between “important” stuff (i.e. the DLR) and “peripheral”
stuff (i.e. IronXxxx). MS wants to have complete control over the DLR and
is not interested in making it Open Source. Rather the DLR code is just
community viewable, much like the rest of the .NET framework code. I can
understand this since core .NET Framework code is central to the MS strategy
and they don’t want things sneaking in the sides.

I disagree. I think DLR is a non-problem. Reality check: do you have
any change in your mind you would like to make to DLR?

For example, (sorry for using CPython as an example; I am not familiar
with Ruby world) many people contributes to CPython runtime without
touching CPython’s custom memory allocator. Still many people
contributes to CPython standard library and C extensions without
touching CPython runtime.

37

On Mon, Apr 28, 2008 at 1:36 PM, Peter Bacon D.
[email protected] wrote:

and they don’t want things sneaking in the sides.

The DLR sources are under the Microsoft Public License as well.

At some point the DLR and the languages will have to be separated,
once the DLR stabilizes to some point. I don’t really think the
current arraignment is viable in the long term, not from an OSS point
of view anyway.

–
Michael L.
[Polymath Prokrammer]
http://blog.prokrams.com

38

Michael L.:

In a technical fashion? No. From an emotional standpoint? Yes.
Right now IronRuby is very unstable from the view of an outside
contributor, you don’t know if the code you’re working on now is going
to need /major/ changes in the next drop, and you don’t know when
that’s going to be. Why work on a bug that “in truth” may already be
fixed?

Agreed. We do maintain our external bug list in Rubyforge which folks
can monitor (are you all receiving update mails on status changes in
tracker?). So you’ll know when we’ve fixed a bug when you see the
Resolution changes from None to Accepted along with some kind of comment
that says ‘fixed in next release’.

The most important change that MSFT can do is let you push to rubyforge
DIRECTLY, none of this internal updates pushed to rubyforge once in a
while. I assume it’s corporate preventing this, because it really make
no sense otherwise. What we have here isn’t an OSS community project
in the traditional sense, what we have is a Microsoft project that
they’ve so kindly, in their infinite wisdom allow us commoners to work
on now and then. Oh but you can’t see or touch the real code until
we’re ready to let you. This is HIGHLY discouraging.

I’ve set the releases traditionally based on whether we had something
‘interesting’ to ship. Sometimes we might go a week or even longer
before substantive changes happen in the Ruby tree. Such is life when
working on compilers - you simply do not check in very often. Remember
that we have Tomas as a full time dev and me as a part time dev on this
project. We’re hiring as well - please send me mail off-list if you’re
interested.

You’ll see more frequent changes in the DLR tree since they have more
devs working on the project.

Thanks,
-John

39

Peter Bacon D.:

I believe one of the key problems is the DLR. As I understand, it MS
makes a distinction between “important” stuff (i.e. the DLR) and
“peripheral” stuff (i.e. IronXxxx). MS wants to have complete control
over the DLR and is not interested in making it Open Source. Rather
the DLR code is just community viewable, much like the rest of the
.NET framework code. I can understand this since core .NET Framework
code is central to the MS strategy and they don’t want things sneaking
in the sides.

Minor correction to this point: The DLR is Open Source in as far as the
license is concerned (which is not like the .NET Framework libraries
source which is released under a much more restrictive read-only
license), which works for folks who are interested in packaging /
redistributing / forking. The DLR is does not accept contributions from
the community, but feedback is certainly welcome.

That said, the bar for feedback is set rather high on the DLR. Most
folks are not compiler implementers, and that’s the feedback that is
needed there. Most folks working on libraries do not need to know
anything about how the DLR is implemented (which is useful since that is
changing rapidly right now). However, if you’re building a language
(we’re doing 3) you have valuable feedback for the DLR team and that’s
one of the goals of IronRuby - to provide feedback to the DLR team.

The reason why the DLR does not accept contributions from the community
is because we intend to ship it in the next version of the .NET
framework. And that means that it ships inside of our commercial
products like Windows. Having community contributions in Windows is
something that we simply cannot do today.

IronRuby, IronPython and so on are not so important to MS strategy and
they are more happy to let the community muck about with the code. I
believe that the long term goal is to open up the IronXxxx code much
more to the community but the problem is that the line between the DLR
and the IronXxxx languages is not yet nailed down. Therefore until
that happens MS is unlikely to hand over the project to the community.

IronPython will move to an accept contributions from the community model
soon.

I would be interested to know how often an SVN dump is created
compared to successful check-ins going through the SNAP process.
Ideally, every successful SNAP check-in should get automatically
dumped out on the RubyForge SVN repos, whether it added value or broke
the tests or whatever. You can always have SVN tags on the “good”
builds and also create downloadable “good” releases on the RubyForge
site - this point would probably help Justin B.'s access problems too.

Today we do not push to SVN on every successful SNAP check-in. That
said, the process on my machine is more-or-less “rake to_svn”, with a
manual check-in after that. I would be more than happy to push on a
daily basis.

Another scenario, which /M:D alludes to if I understand correctly, is
to allow the community to modify the code in the RubyForge project and
then let MS select “good” builds to check back into the Team system
via the SNAP process. That way, the community feels ownership of the
project and MS get that quality control on what finally goes into
IronRuby.
There are obviously many technical hurdles to overcome before this
could become reality. In particular, there needs to be a separation
of DLR from IronXxxx, including, probably, some kind of stable release
of the DLR for the IronXxxx projects to work off.

Exactly. Right now the integrated model is good since it means that we
progress faster. In the future, when we move to a modular model, it
means that DLR changes will break IronRuby which means more work for
everyone on this end (the DLR devs will be happier since they’ll spend
less time fixing us). The bottom line is that work is generated - it’s
just who feels the pain.

Any other ideas? John what are you thinking here?

I collected my thoughts in the other thread that I started last night.
Thanks for your ideas!

-John

40

Peter Bacon D.:

My point wasn’t that one needs to have access to the DLR code. It was
that because IronRuby is so tightly coupled to DLR at the moment, it is
not possible to remove its tethers and let it free as a proper OSS.
Pete

Yep. BTW, this is exactly the set of arguments that are used in making
the distinction between integrated and modular systems. We are
integrated now because we are optimizing for finishing our work on the
DLR (and the first set of languages) sooner. We will become modular in
the future because it lets folks build on top of us more easily.

If you’re curious listen to Clay Christensen’s most excellent thoughts
on this topic in this podcast:

-John