I seem to have been missing in action for the best part of six months so
I thought it was about time I put together a progress report and pimped
the vapourware in hopes of finding some likeminded
miscreants/dilettantes to turn it into something more substantial.
There’re probably only a handful of people on here who’ll be interested
in what follows so I’ve marked it OT for those of you interested
exclusively in Ruby. Feel free to press delete, I won’t be offended 
== BACKGROUND ==
For those of you without long memories, a quick recap. At the tail-end
of last year Google released their Go language (http://www.golang.org)
onto an unsuspecting world. The brainchild of Rob Pike and Ken Thompson,
Go’s sort of a reworking of Limbo from Plan 9/Inferno with baked-in
concurrency and an inside-out approach to object orientation.
As those of you who’ve had the dubious pleasure of either sharing bar
space with me or sitting through one of my conference sessions on Ruby &
Unix, for all that Ruby’s my language of choice I still spend a fair
amount of time grubbing around in the C world. Love them or loathe them,
there are times when we all need systems-level languages to efficiently
solve certain kinds of problems.
The longterm goal of RubyGoLightly is to make Ruby a first-class citizen
in that sense, moving systems coding firmly and permanently into our
high-level world. I know it can be done as Ruby’s pretty much as
flexible as Lisp and Symbolics proved that an OS could be written in
Lisp with minimal dependence on assembly language. So the aspiration
isn’t so lousy even if I’m far from certain that I’m personally capable
of achieving it.
There is however one major roadblock to turning Ruby into a systems
language: existing runtimes just aren’t written that way. The closest we
have at present is JRuby, and even there if you want complete control
you still have to step outside of the Ruby mindset and hack Java, JVM
bytecode or probably even write JNI extensions in C.
Then there’s the question of an optimal Ruby runtime. There’s a lot of
implicit concurrency in the average Ruby program which to the best of my
knowledge isn’t being addressed by any of the existing implementations.
We live in a multicore world where it’s only natural that collection
enumerators etc. should be exploiting those cores for cheap scalability.
Admittedly writing concurrent systems in C can be a right pain in the
arse, and even in Java there are subtleties arising from the
architecture of the JVM so I’m not levelling criticism at anyone. It’s
just a fair observation that if I have four processor cores and a list
with 10K entries that map/reduce algorithms should use all four cores if
at all possible and return their results in approximately 1/3rd the time
of a naive sequential implementation.
Given that Intel are playing with 80+ core processors in the lab - and
that modern graphics cores have even more available horsepower via
OpenCL - there’s clearly an opportunity worth investigating.
Go does concurrency pretty well, especially considering that the runtime
is still in its infancy and will probably undergo considerable
optimisation in the future. It’s basically an implementation of Hoare’s
CSP so all the tricks that can be used in Occam etc. for clean
concurrency can be used in Go, and whilst it doesn’t yet support OpenCL
that’s just a matter of time and a sufficiently nasty itch being worth
scratching.
Another desirable trait for a Ruby runtime is decent browser
integration. I get really annoyed having to write heavyweight
client-side web code in JavaScript as to me it’s an ugly language. I’ve
probably never recovered from my painful experiences with NewtonScript
as I find the entire Self language family unbearable, and the same goes
for the slot-based GTK GUI stuff (sorry webGTK). Go has support for
Google’s Native Client so is ideal for injecting native code into web
content, including a Ruby runtime were it to exist. I love that idea.
Write your whole web app in Ruby, with identical abstractions.
The same thing could be achieved using a Ruby runtime written in
JavaScript (such as HotRuby) but to date that approach has stalled.
Basically writing any significant runtime platform is a significant cost
and taking anything beyond the complexity of a simple bytecode
interpreter means months of full-time hacking and serious cogitation.
The final thing in Go’s favour is that it can compile to native code for
bare metal - i.e. no OS at all, just a bootloader and an executable.
I’ve yet to play with that side of the language, but having written
assembler for embedded systems with no OS I would love to see a Ruby
runtime deployed in the same manner. From there it’s a short step to an
OS written in Ruby, and that means an OS any half-competent programmer
could understand in detail.
My colleague Romek and I have been boring people with that idea for
about five years now, and mad though it probably seems I still believe
it’s a goal worth pursuing.
Anyway, to recap: the world might be a better place with a Ruby runtime
that embraced pervasive concurrency in the runtime implementation, that
was platform agnostic, and that allowed systems-level coding to be
performed with the same ease we associate with Rails or even Sinatra. I
won’t pretend RubyGoLightly will tick all these boxes - right now it’s
an idea and ideas are by their nature transient. However it is certainly
a framework in which to explore whether such a Ruby is possible.
== GoLightly ==
There are two aspects to this project. One is the Ruby runtime, which
right now I have only the vaguest of ideas about how to implement. I’m
following a hunch that a Forth-like Threaded Interpreter might allow
optimisations that we’re not going to see with mainstream bytecode
interpreters and a lot of current effort is devoted to writing such a
Forth implementation. Although I hesitate to call it Forth given the
usual expectations of F79, F83, figForth or ANS94 compliance - none of
them particular of particular importance.
Threaded interpretation isn’t a million miles removed from the naive
view of Ruby’s object model so I’m hoping my current work will generate
insights relevant to that, however its main purpose is to drive design
of the GoLightly virtual machine which is an abstraction of the kind of
hardware I’m interested in seeing Ruby run on.
There’s an early branch of GoLightly on github
(http://github.com/feyeleanor/GoLightly) which exposes some basic
despatch loops (inlined bytecode and indirect function call) and a
preliminary model for SIMD-style opcodes for basic vector processing and
a supporting register model.
There are a few examples of hand-coded assembler in the tests to give a
feel for the machine language along with some micro-benchmarks for
anyone who fancies playing. GoLightly already copes with multiple
virtual processors, all working on their discrete problems but there’s a
lot of work yet to be done in inter-core communication, the memory
allocation model, and parallelising SIMD instructions.
One of the few things I dislike about Go as it currently stands - and
which alas is considered a strength by the core team so is unlikely to
change - is that the language doesn’t support a representation-agnostic
datatype suitable for holding bit-strings: basically numeric types
always convert their contents to suit the type to which they’re being
coerced, or else a panic occurs. Getting around this involves playing
games with pointers that make a fair amount of (void *) usage in C look
clear and reasonable. The justification is that this sort of thing
shouldn’t be encouraged (with which I happen to disagree anyway) and
that the extra pain is a bit of nudge to prevent it. I’ll keep raising
the issue with the core team every few weeks as I personally believe
maintainability is much more important than type safety: after all, we
can always write tests…
Anyway GoLightly is stalled whilst I evolve a reasonable API for
handling bitstring manipulations, which is proving a subtly intractable
problem. I’m hoping that hacking together a few languages will help me
figure out most of the implementation details which are currently
eluding me and then I’ll roll a bitstring library back into the Go core
library if the team will have it.
The unnamed Forth I’m currently working on is running into the same
problems, but I’m still hoping to get a reasonable build ready for
release on github over the summer. So far I have a good first cut of a
compile/execute/decompile model which supports dynamic coding and I hope
to reduce much of the Go code for implementing a core vocabulary to
GoLightly-style bytecode. These bytecodes (which are effectively unique
to the currently running system, so more a dynamic object code than a
formal bytecode) would themselves be a first-class element of the
language so that’s an interesting model for both a native code interface
and for allowing inline use of multiple languages.
== The Roadmap ==
From Forth to Ruby is one hell of a leap, and whilst I don’t doubt it
can be done in one step there are other problems I want to solve along
the way.
I definitely see a simple Lisp being in the mix as I’ve always felt
there is an equivalence between Forth and Lisp which goes unremarked: I
guess there aren’t that many people who’ve hacked with both, let alone
who’ve written runtimes for both. Lisp will be the testbed for doing
standard function call stack allocations and a few other aspects of
implementing a high-level language which Forth skirts. Common Lisp it
most definitely won’t be though (or Arc for that matter).
From there I want to get a Scheme working, mostly to figure out how to
handle tail-call recursion efficiently. It could be that this will be a
specific compiler optimisation required separately for each supported
language, but I’d really prefer it to work at the object code level so
that it’s applied across all compiled code automatically. Go uses
relatively small stacks to improve concurrent performance compared to
pThreads and whilst GoLightly threads are currently specced to use their
own user-space threads I’d like to move beyond that in the future so
tail call culling will be essential to not swamping the runtime stack.
After Scheme I’m hoping to implement a simple Logo system to get a
handle on the interfaces offered by Google’s Native Client etc. - it’s a
bit of a gratuitous diversion as Logo is hardly a mainstream language,
but the underlying structure is similar enough to Lisp that I think it
will be an easy build and there’s been some interesting work in
Logo-land to do with multi-threaded graphics which might have
interesting tie-ins with OpenCL.
Much of the impetus to start hacking on RubyGoLightly is thanks to
TinyRb, a really sweet subset of Ruby running on a Lua-style VM. Once
I’ve solved the main runtime problems addressed by the Lisp arc of
languages I plan to write a runtime interpreter for the TinyRb bytecode
so its compiled form can run natively in GoLightly. My gut instinct is
that this will be a great starting point for runtime optimisations that
apply to a richer subset of Ruby.
At LSRC last year I showed off some of the code from a port of SQLite to
Ruby that I was working on at the time. That project got put on hold
when I started RubyGoLightly but given that SQLite is a really sweet
engine and internally uses a register based VM it’d be easy to port to
GoLightly and provide something like CoreData out of the box.
There could be a few steps after this: for one thing I’ve this gut
feeling that doing a dirty implementation of Io could be instructive,
though I can’t really explain way. Something to do with the method
passing semantics. A roadmap’s no good if it doesn’t allow for
interesting detours
However the ultimate destination is RubyGoLightly, the Ruby runtime that
scratches the itch of using Ruby as a systems language. No promises
about where or when, and very few about how… but I promise to check in
with another status report in a few months that will hopefully be more
insightful.
In the meantime, happy hacking,
Ellie
Eleanor McHugh
Games With Brains
http://feyeleanor.tel
raise ArgumentError unless @reality.responds_to? :reason
