How can we make a ruby compiler

Ruby
41

On 5/10/2010 8:02 AM, Tony A. wrote:

Perhaps the most straightforward answer for wanting a Hiphop or
Starkiller-like compiler is FOR SPEED! For example, you’re a company on the
scale of Facebook who has written a large web site in a dynamic language and
suddenly realize you could be saving a lot of money on servers if your app
ran a lot faster. Source translation to C++ is a potential way to go

I don’t think that applies to anyone who uses Ruby, though. Maybe Twitter…

If you get to that scale (and are hopefully making money by then or have
very large investments that match your scale of users/ traffic), there
is a high chance that you will need to re-architect or rewrite a bunch
of stuff - I don’t think source translation will solve the problem.

On the other hand, Speed is an objective is something I am very keen
on… I think there’s a bunch of stuff that we do which is acceptable in
Ruby but I would prefer it to run faster in some cases. However, I have
often found that Ruby lets me do the first prototype fast and once the
system is mature, if the speed is needed (not desired), a rewrite in C++
is not that difficult - it should be noted that I’m not a super-advanced
Ruby programmer… what that means is that I tend to write Ruby code
that avoids the advanced constructs of Ruby, making translation to C++
easier.

Cheers,
Mohit.
5/10/2010 | 5:10 PM.

42

On Wed, Oct 6, 2010 at 12:13 AM, Tony A. [email protected]
wrote:

would preclude many of the things which make Ruby fun and interesting.
This is very true. Dipping into what seems to have been attempted in
this area reveals lots of attempts which solve a small but easy subset
of the problem, like translating a “hello world” program, or a small
numerical problem with some polymorphic numeric values from Ruby or
Python to C or C++, and then stall out either when the hard problems
arise, or when the requirements to qualify for a Master’s degree
project are met.

And Mohit’s original observation is also important:

If you get to that scale (and are hopefully making money by then or have very large investments that
match your scale of users/ traffic), there is a high chance that you will need to re-architect
or rewrite a bunch of stuff - I don’t think source translation will solve the problem.

Languages differ in much more important and subtle ways than their
syntax and surface-level semantics. Those differences lead to very
different architectures/strategies or whatever you want to call it.

Converting an OO program written in, say Ruby, requires adaptation of
not only the language syntax, but the underlying class library, if you
are converting to C++, you need to figure out what to do about the
usage in the Ruby program of base classes, you do probably have
several choices of C++ libraries providing things like collection
classes, all of which will mismatch the interfaces of their Ruby
equivalents, probably in frightening ways.

Replace Smalltalk/Ruby message send semantics with C++ virtual
function calls, and the constraints change. Move the burden of
accounting for and reclaiming the memory used by objects from a GC to
the programmer and things get complicated, and often the performance
of the hand-written memory management (copy constructors, reference
counts …) starts making the performance ‘win’ of switching horses
much less certain.

This is Greenspun’s tenth rule of programming at work.
http://philip.greenspun.com/research/

–
Rick DeNatale

Blog: http://talklikeaduck.denhaven2.com/
Github: rubyredrick (Rick DeNatale) · GitHub
Twitter: @RickDeNatale
WWR: http://www.workingwithrails.com/person/9021-rick-denatale
LinkedIn: http://www.linkedin.com/in/rickdenatale

43

Eleanor McHugh wrote:

On 5 Oct 2010, at 22:07, William R. wrote:

As for a Ruby compiler, a useful one is very unlikely for many reasons. Now a compiler for a different language that has a distant family resemblance to Ruby may be possible but likely of not much practical use.

Lisp, Scheme, Forth and Smalltalk are all compilable so in principle Ruby should be as well. It’s just not clear that there’s any real gain by doing so outside of very particular applications (i.e. dominated by maths or interpretation performance bottlenecks). Moreover multicore is teasing us towards a world where all processing will be a negligible cost compared to I/O latencies and throughput, much as was the case fifty years ago.

I have just reread all the messages in this thread and decided not to
finish a draft reply that was already too long. In summary, I argued
that possible language implementations lie on a continuum with “pure”
compilers and interpreters at the extremes, and most implementations for
real languages somewhere in the middle. Therefore, it is not very
productive to discuss whether a Ruby compiler is possible, but instead
focus on solutions to the specific problems that cause people to ask
about compilers for Ruby.

In this thread, only Philip R. has mentioned a real application
that he would like to be able to reprogram in Ruby and still have run
times comparable to C. Perhaps he can expand, preferably in a new
thread, on his program and the benefits he hopes to achieve by moving
the work to Ruby.

– Bill

44

On Tue, Oct 5, 2010 at 9:54 PM, Mohit S. [email protected]
wrote:

I tend to write Ruby code that avoids the advanced constructs of Ruby,
making translation to C++ easier.

This is pretty much a staple of static type inferencers for dynamic
languages that output C++… many language features simply cannot be
supported using this approach, most notably eval, and eval is pretty
much
the most powerful tool for Ruby metaprogramming. Source translation to
C++
would preclude many of the things which make Ruby fun and interesting.

45

On Wed, Oct 6, 2010 at 9:11 AM, William R. [email protected]
wrote:

In this thread, only Philip R. has mentioned a real application that he
would like to be able to reprogram in Ruby and still have run times
comparable to C. Perhaps he can expand, preferably in a new thread, on his
program and the benefits he hopes to achieve by moving the work to Ruby.

Just to clarify, here’s what he originally wrote:

I have an actual existing application (C/C++) for population genetics simulations
which I would dearly love to convert to Ruby.
It was originally all in C and, as a learning exercise, I converted parts of it to C++ - but it was such a pain .
. it would have been so pleasant to re-write in Ruby.
However I would have to get resulting code converted back to C or compiled somehow to
get the performance back to something usable.

It’s not clear from this whether he has actually tried to recode the
program in Ruby or this is still a gedanken experiment.

The first step would be to try to re-code in Ruby and see where the
performance bottlenecks resulting from the conversion, if any, were.
Then optimizing these steps either by rewriting the Ruby or converting
selective code to C extensions might get him to the performance he
seeks, or at least acceptable performance.

Rick DeNatale

Blog: http://talklikeaduck.denhaven2.com/
Github: rubyredrick (Rick DeNatale) · GitHub
Twitter: @RickDeNatale
WWR: http://www.workingwithrails.com/person/9021-rick-denatale
LinkedIn: http://www.linkedin.com/in/rickdenatale

46

On 6 Oct 2010, at 13:11, Rick DeNatale wrote:

equivalents, probably in frightening ways.
Philip Greenspun's Research
Funnily enough this is one of the inspirations behind my golightly[0]
project: if every significant program I write will end up with an ad hoc
implementation of Lisp (or more often than not Lisp and several other
languages) then why not just use an abstract platform that allows me to
plug together Lisp machines, Forth machines, Ruby machines etc. and have
them all operate on the same underlying type space?

This is subtly different to the traditional virtual machine approach as
by taking seriously the notion of language agnosticism you end up with
less a single machine and more a set of components that can be plugged
together to build an infinite variety of machines that are each better
suited to the particular tasks they’re used for than a general-purpose
VM. I suspect it’s probably similar to the software IC concept you
mentioned over on Core recently, but with the added wrinkle that I’m
interested in building in generalised type space transformations if I
can get my head around them. These would effectively provide a means of
JIT translation between different language type models, including
assembler-level type expression where practical. I suspect a big chunk
of the winter is likely to be spent researching current developments in
compiler theory in hopes someone else has already solved this problem…
and if they haven’t perhaps it’s time I went back to Uni and got my
Doctorate :slight_smile:

Ellie

[0] http://golightly.games-with-brains.net/

Eleanor McHugh
Games With Brains
http://feyeleanor.tel

raise ArgumentError unless @reality.responds_to? :reason

47

Ruby can be dynamically translated ( = jitted ) as Smalltalk is.
A project “Rubinius” exists which goes in that direction.
The problem with ruby is that it is not so easy ( = lot of work ) to
write a
ruby to bytecode translator.
Smalltalk & List style languages are almost syntax free - the contrary
is
the case for Ruby.
Or Smalltalk VM ( LSW-GVM ) runs Smalltalk at the speed of C/C++ and we
plan
to do that also for other dynamic languages ( e.g Haskell, JavaScript
also
Ruby ) in the nearest future. The LSW-GVM is currently available for
Win32
( deep GUI integration, multithreading ). We are working on a 64 Bit
version.

Frank Lesser

“Tony A.” [email protected] schrieb im Newsbeitrag
news:AANLkTinzm=ch-Vz8T6zFAa_xMoZq=removed_email_address@domain.invalid…

48

On Wed, Oct 6, 2010 at 9:56 AM, Eleanor McHugh
[email protected] wrote:

On 6 Oct 2010, at 13:11, Rick DeNatale wrote:

This is Greenspun’s tenth rule of programming at work.
Philip Greenspun's Research

Funnily enough this is one of the inspirations behind my golightly[0] project: if every significant program I write will end up with an ad hoc implementation of Lisp (or more often than not Lisp and several other languages) then why not just use an abstract platform that allows me to plug together Lisp machines, Forth machines, Ruby machines etc. and have them all operate on the same underlying type space?

This is subtly different to the traditional virtual machine approach as by taking seriously the notion of language agnosticism you end up with less a single machine and more a set of components that can be plugged together to build an infinite variety of machines that are each better suited to the particular tasks they’re used for than a general-purpose VM. I suspect it’s probably similar to the software IC concept you mentioned over on Core recently, but with the added wrinkle that I’m interested in building in generalised type space transformations if I can get my head around them. These would effectively provide a means of JIT translation between different language type models, including assembler-level type expression where practical. I suspect a big chunk of the winter is likely to be spent researching current developments in compiler theory in hopes someone else has already solved this problem… and if they haven’t perhaps it’s time I went back to Uni and got my Doctorate :slight_smile:

Sounds a bit like several historic efforts like Uncol, IBM’s SOM, and
Microsoft’s (D)CLR.

Also sounds for a great topic for a Ruby Conf version of the kind of
semi-inebriated late night discussions I used to enjoy at OOPSLA with
the likes of Kent Beck, Ward Cunningham, George Bosworth of Digitalk
(who went on to work on DCLR at MS before giving up on them) and
others back in the heyday.

–
Rick DeNatale

Help fund my talk at Ruby Conf 2010:http://pledgie.com/campaigns/13677
Blog: http://talklikeaduck.denhaven2.com/
Github: rubyredrick (Rick DeNatale) · GitHub
Twitter: @RickDeNatale
WWR: http://www.workingwithrails.com/person/9021-rick-denatale
LinkedIn: http://www.linkedin.com/in/rickdenatale

49

On 6 Oct 2010, at 15:48, Rick DeNatale wrote:

Sounds a bit like several historic efforts like Uncol, IBM’s SOM, and
Microsoft’s (D)CLR.

I don’t think anyone’s done this particularly well to date, in large
part because it’s a ferociously difficult problem domain to get a decent
handle on. But I guess that’s part of the attraction lol

Also sounds for a great topic for a Ruby Conf version of the kind of
semi-inebriated late night discussions I used to enjoy at OOPSLA with
the likes of Kent Beck, Ward Cunningham, George Bosworth of Digitalk
(who went on to work on DCLR at MS before giving up on them) and
others back in the heyday.

Well I’ve never been backward in geeking when there’s beer on tap,
though finding anyone who can understand what the hell I’m on about is
always a bit of a sticking point. They just don’t make young geeks like
they used to…

Ellie

Eleanor McHugh
Games With Brains
http://feyeleanor.tel

raise ArgumentError unless @reality.responds_to? :reason

50

I may be wrong, but I’ve read that IronRuby can be set to allow
compilation after x loops through a process, in order to speed up
execution.

Shall that be a (possible) solution to the puzzle?
Specially if that “compiled” code can be saved somewhere?

I’m not that technically savvy, however you can check with the great
guys in the IronRuby forum about it.