Code Review: RubyLibsAndYaml

Ruby IronRuby
1

tfpt review /shelveset:RubyLibsAndYaml;REDMOND\tomat

Adjusts class-initializer generator to enable generating initializers
for multiple libraries within a single assembly.
Also enables extending existing Ruby modules and classes that are
defined in C# libraries (previously only CLR classes could have been
extended). For example, the following class adds additional methods on
Kernel module:

[RubyModule(Extends = typeof(Kernel))]
public static class MyKernelOps {
[RubyMethod(“foo”)]
public static void Foo() { … }
}

The type specified in Extends parameter is the C# class defining the
module/class.

Enables to group C# classes and modules into Ruby libraries (e.g.
thread, socket, openssl, yaml, etc.). Each library is identified by a
root namespace. All Ruby classes and modules of the assembly defined
within the namespace are considered parts of the library. The shelveset
updates namespaces in IronRuby.Libraries.dll to group classes by
library:

Ruby.Builtins
Ruby.StandardLibrary.Threading
Ruby.StandardLibrary.Sockets
Ruby.StandardLibrary.OpenSsl
Ruby.StandardLibrary.Digest
Ruby.StandardLibrary.Zlib

The list of namespaces that define libraries within an assembly is
passed to the generator:

ClassInitGenerator IronRuby.Libraries.dll
/libraries:Ruby.Builtins;Ruby.StandardLibrary.Threading;Ruby.StandardLibrary.Sockets;Ruby.StandardLibrary.OpenSsl;Ruby.StandardLibrary.Digest;Ruby.StandardLibrary.Zlib
/out:Initializers.Generated.cs

The mapping from Ruby library name to the assembly and namespace is
established via .rb files:

thread.rb:
load_assembly ‘IronRuby.Libraries’, ‘Ruby.StandardLibrary.Threading’

openssl.rb:
load_assembly ‘IronRuby.Libraries’, ‘Ruby.StandardLibrary.OpenSsl’

etc.

(Kernel#load_assembly now takes an optional second argument identifying
the library to load by its root namespace).

The files are included in the solution and their build action is set to
“copy if new”, which means they are copied to the output directory
during the build. ir.exe includes the directory it is located in into
the load paths list ($:), so the .rb files copied there are found by the
standard Ruby require/load mechanism.

Tomas

2

openssl.rb:
load_assembly ‘IronRuby.Libraries’, ‘Ruby.StandardLibrary.OpenSsl’

The files are included in the solution and their build action is set to “copy if new”, which means they
are copied to the output directory during the build. ir.exe includes the directory it is located in into the
load paths list ($:), so the .rb files copied there are found by the standard Ruby require/load mechanism.

So what if some legacy code requires ‘thread.so’ or ‘opensll.so’? Can we
set it up so that it automatically loads the corresponding library?

3

Wayne K.:

to “copy if new”, which means they are copied to the output directory
during the build. ir.exe includes the directory it is located in into
the load paths list ($:), so the .rb files copied there are found by
the standard Ruby require/load mechanism.

So what if some legacy code requires ‘thread.so’ or ‘opensll.so’? Can
we set it up so that it automatically loads the corresponding library?

I’m not sure what the best solution to this problem is here - we should
probably throw an exception if *.so/.dll does not refer to a .NET
assembly. It’s unlikely that existing code will require thread.so rather
than thread.rb. Charlie - what does JRuby do in this case?

Thanks,
-John

4

John L. (IRONRUBY) wrote:

The files are included in the solution and their build action is set
to “copy if new”, which means they are copied to the output directory
during the build. ir.exe includes the directory it is located in into
the load paths list ($:), so the .rb files copied there are found by
the standard Ruby require/load mechanism.

So what if some legacy code requires ‘thread.so’ or ‘opensll.so’? Can
we set it up so that it automatically loads the corresponding library?

I’m not sure what the best solution to this problem is here - we should probably throw an exception if *.so/.dll does not refer to a .NET assembly. It’s unlikely that existing code will require thread.so rather than thread.rb. Charlie - what does JRuby do in this case?

In Ruby, load path searching is partially determined by the extension.
If you try to require a file with no extension, it looks first for that
file with a .rb extension, then for that filename as a .so library (or
.dll, or whatever the library extension is). If you try to load a file
with no extension, it will only ever search for exactly that file with
no extension.

In JRuby, we just use the .so/.dll as a marker for searching for
extensions, skipping the search for .rb. And in the cases of
thread.rb/thread.so, we just have a very small thread.rb file that loads
our actual extension, thread.so. In fact, most of the code extensions we
have defined to be .so, .rb, or sometimes both so that any load will
find them.

  • Charlie
5

For that case, we may need to add file called ‘thread.so.rb’.

Tomas

6

-----Original Message-----
From: John L. (IRONRUBY)
Sent: Monday, 19 May 2008 6:21 AM
To: [email protected]
Subject: Re: [Ironruby-core] Code Review: RubyLibsAndYaml

I’m not sure what the best solution to this problem is here -
we should probably throw an exception if *.so/.dll does not
refer to a .NET assembly. It’s unlikely that existing code
will require thread.so rather than thread.rb.

I was primarily referring to:

ruby\lib\ruby\1.8\thread.rb which explicitly loads thread.so
ruby\lib\ruby\1.8\openssl.rb which explicitly loads openssl.so
ruby\lib\ruby\1.8\digest.rb which explicilty loads digest.so
ruby\lib\ruby\1.8\pp.rb which explicitly loads etc.so

These are all part of the Windows One click installer distribution.

The .so files reside in ruby\lib\ruby\1.8\i386-mswin32

I’ve been assuming that it is OK to alter files in this i386-mswin32
directory as they are intended to be platform specific implementations.
As IronRuby is effectively a different platform, I’ve been assuming that
we will rename this directory ‘ironruby’. I was also assuming, however,
that in the long run, it would be “cheating” to modify any of the other
ruby source files in the standard libraries in order to get IronRuby to
work correctly.

So, what are the ground rules for altering these libraries?

I assume we don’t want to get into the business of packaging and
distributing the ruby source files in these standard libraries
ourselves.

Charles, to what extent does JRuby stick with the existing MRI ruby
library directory hierarchy?

On a related issue - MRI automatically includes a number of relative
library paths into its load search path. Currently in IronRuby we need
to explicitly list each of these libraries in the command line using the
/paths command line option (and it’s really painfull when there’s half a
dozen long paths included on your command line). Is there any plan to
add any of these search paths automatically, or at least semi
automatically? We may want to provide a mechanism for specifying the
standard library root (eg “c:\ruby\lib”) rather than assuming that it is
relative to the ruby interpreter executable, but couldn’t we then
automatically add other “standard” paths relative to that?

There is of course also the possibility of creating a new (additional)
IronRuby console front end that behaves more like MRI in terms of
command line options.

Cheers, Wayne.

7

Hey folks,

Does there exist a formal code review process for the Iron* languages
and/or the DLR? Have you considered using a tool to ease gathering of
review feedback?

http://www.review-board.org/ - VMWare, Christian Hammond
http://reviews.review-board.org/dashboard/

svn checkout http://reviewboard.googlecode.com/svn/trunk/reviewboard

http://codereview.appspot.com/ - Google, Guido van Rossum

Cheers,

C.J.

8

Wayne K. wrote:

Charles, to what extent does JRuby stick with the existing MRI ruby library directory hierarchy?

We use the basic structure for .rb files, but we do not put extensions
on the filesystem anywhere since all of them are bundled into JRuby’s
main JAR file. We have hooks in the loading process to find extensions,
however, and gem-based extensions can load their own libraries from
within the unpacked gem location through a number of mechanisms.

  • Charlie
9

Wayne K.:

directory as they are intended to be platform specific implementations.
As IronRuby is effectively a different platform, I’ve been assuming
that we will rename this directory ‘ironruby’. I was also assuming,
however, that in the long run, it would be “cheating” to modify any of
the other ruby source files in the standard libraries in order to get
IronRuby to work correctly.

So, what are the ground rules for altering these libraries?

I have no idea right now. That said, I think we’re going to need a
process for filtering out a bunch of libraries that don’t make sense for
our implementation. For example, the various Tcl related libraries are a
reasonable candidate to remove. So we’re already going down the path of
modifying the sate of lib\ruby\1.8.

Since those libraries you referenced above are nothing more than just
wrappers around loading native libraries, I don’t see the harm in
changing them. We could also use Tomas’ hack around creating a
thread.so.rb file to thunk to loading thread.so as well …

On a related issue - MRI automatically includes a number of relative
library paths into its load search path. Currently in IronRuby we need
to explicitly list each of these libraries in the command line using
the /paths command line option (and it’s really painfull when there’s
half a dozen long paths included on your command line). Is there any
plan to add any of these search paths automatically, or at least semi
automatically? We may want to provide a mechanism for specifying the
standard library root (eg “c:\ruby\lib”) rather than assuming that it
is relative to the ruby interpreter executable, but couldn’t we then
automatically add other “standard” paths relative to that?

I added some time ago an implementation for -I on the Ruby command line.
I have the following PowerShell alias defined:

function rbd { cmd /C “${env:MERLIN_ROOT}\Bin\Debug\ir.exe -I
${env:MERLIN_ROOT}…\External\Languages\Ruby\ruby-1.8.6\lib\ruby\1.8;${env:MERLIN_ROOT}…\External\Languages\Ruby\ruby-1.8.6\lib\ruby\site_ruby\1.8;${env:MERLIN_ROOT}…\External\Languages\Ruby\ruby-1.8.6\lib\ruby\1.8\i386-mswin32
-D $args”}

You should be able to do the same with your local copy.

There is of course also the possibility of creating a new (additional)
IronRuby console front end that behaves more like MRI in terms of
command line options.

The goal is to make ir.exe do exactly that. We just haven’t invested
beyond -I today.

Thanks,
-John

10

C.J. Adams-Collier:

Does there exist a formal code review process for the Iron* languages
and/or the DLR? Have you considered using a tool to ease gathering of
review feedback?

http://www.review-board.org/ - VMWare, Christian Hammond
http://reviews.review-board.org/dashboard/

svn checkout http://reviewboard.googlecode.com/svn/trunk/reviewboard

http://codereview.appspot.com/ - Google, Guido van Rossum

We really should do this - the existing method of patches is definitely
sub-optimal. But we have a problem with folks using two completely
different environments to do their coding. We’ll fix this problem
post-RailsConf/Tech-Ed.

Thanks,
-John

11

I couldn’t find the source for Rietveld before sending the first mail.
Guido was kind enough to point me to it, though:

Cheers,

C.J.