Forum: Ruby [ANN] duby 0.0.1 Released

duby version 0.0.1 has been released!

* <http://kenai.com/projects/duby>

Duby is a customizable programming language featuring static types,
local type inference and a heavily Ruby-inspired syntax. Duby
currently includes a typer/compiler backend for the JVM which can
output either JVM bytecode or Java source files.

Changes:

### 0.0.1 / 2009-10-27

* First gem release
  * Most things necessary for simple apps, scripts, servlets are working

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

From: "Charles Oliver Nutter" <headius@headius.com>
>
> duby version 0.0.1 has been released!
> 
> * <http://kenai.com/projects/duby>
> 
> Duby is a customizable programming language featuring static types,
> local type inference and a heavily Ruby-inspired syntax. Duby
> currently includes a typer/compiler backend for the JVM which can
> output either JVM bytecode or Java source files.

Wow!  That Fibonacci example on 
http://kenai.com/projects/duby/pages/DubySamples
is really neat, especially the return value inferencing.

I would love it if ruby had something baked-in to the language that was
so unobtrusive, so much like writing regular ruby, but which could
compile to fast static code.

I've wondered to what degree such a thing might be possible, but I
never realized it could really mirror regular ruby that closely.  Very 
cool.

Is your system able to compile floating point operations down to
primitives the way it does with fixnums?


Regards,

Bill

On Tue, Oct 27, 2009 at 5:13 PM, Bill Kelly <billk@cts.com> wrote:
> Wow!  That Fibonacci example on
> http://kenai.com/projects/duby/pages/DubySamples
> is really neat, especially the return value inferencing.
> I would love it if ruby had something baked-in to the language that was
> so unobtrusive, so much like writing regular ruby, but which could compile
> to fast static code.
>
> I've wondered to what degree such a thing might be possible, but I
> never realized it could really mirror regular ruby that closely.  Very cool.

I'm very interested in adding optional static typing to JRuby, for
people that need it. Duby is, in a way, research into how this can be
done without damaging Ruby's syntax substantially. I think the
trade-offs so far in Duby are acceptable.

Granted, some people will want to lynch me for even suggesting the
idea. But when you need static types, or the performance that can come
more easily from static types, it's nice to have without dropping to C
or Java. So it's worth exploring for Ruby, and JRuby is the Ruby I
know how to hack.

> Is your system able to compile floating point operations down to
> primitives the way it does with fixnums?

Yes, check the bench_fractal.duby benchmark in examples/. It's mostly
the same code as the Ruby version, but runs almost two OOM faster.
Heavy floating-point math.

And I should make it clear....Duby is not a "statically typed Ruby".
It's a different language that co-opts Ruby's syntax and adds static
types and uses Java/JVM type system (though other backends are
possible).

- Charlie

On Tue, Oct 27, 2009 at 5:13 PM, Bill Kelly <billk@cts.com> wrote:
> Wow!  That Fibonacci example on
> http://kenai.com/projects/duby/pages/DubySamples
> is really neat, especially the return value inferencing.

Added the fib example plus fractal benchmark and a simple Swing
example to http://kenai.com/projects/duby/pages/Examples

- Charlie

On Oct 27, 2009, at 15:13 , Bill Kelly wrote:

> Wow!  That Fibonacci example onhttp://kenai.com/projects/duby/pages/DubySamples
> is really neat, especially the return value inferencing.
> I would love it if ruby had something baked-in to the language that  
> was
> so unobtrusive, so much like writing regular ruby, but which could  
> compile to fast static code.

You do NOT need static code for it to be fast. See smalltalk, self,
and many many other examples.

Ruby's method dispatch is pathologically slow.

Ryan Davis <ryand-ruby@zenspider.com> writes:

> and many many other examples.
>
> Ruby's method dispatch is pathologically slow.

"Fast" is a relative term, and I'd agree that Ruby can be faster w/o
resorting to static typing, but it does seem that statically typed
languages such as Haskell, OCaml, C, Java, etc. have a performance
edge over dynamically typed languages in general.

Or it could be that I'm just not aware of dynamically typed languages
that are faster than the statically typed ones listed above.

On Tue, Oct 27, 2009 at 5:29 PM, Ryan Davis <ryand-ruby@zenspider.com> 
wrote:
> You do NOT need static code for it to be fast. See smalltalk, self, and many
> many other examples.

Sure, it's just easier with static types. Don't like it, don't use it.

- Charlie

Glad to hear about this, and look forward to using it on a future 
project :)

On Tue, Oct 27, 2009 at 3:54 PM, Charles Oliver Nutter

Charlie,

Your Strange Loop talk compared examples in Ruby, Duby, and Surinx. It 
was
interesting how the examples differed very little from each other. I 
wonder
if automated translation between the three would be possible? Perhaps 
the
only realistic direction would be from Duby/Surinx -> Ruby (e.g., 
removing
type information).

Also, have you documented the syntax differences somewhere, in case 
someone
wanted to do a manual translation?

dean


On Tue, Oct 27, 2009 at 5:19 PM, Charles Oliver Nutter
<headius@headius.com>wrote:

> > never realized it could really mirror regular ruby that closely.  Very
> or Java. So it's worth exploring for Ruby, and JRuby is the Ruby I
> It's a different language that co-opts Ruby's syntax and adds static
> types and uses Java/JVM type system (though other backends are
> possible).
>
> - Charlie
>
>


--
Dean Wampler
coauthor of "Programming Scala" (O'Reilly)
-  http://programmingscala.com

twitter: @deanwampler, @chicagoscala
Chicago-Area Scala Enthusiasts (CASE):
-  http://groups.google.com/group/chicagoscala
-  http://www.meetup.com/chicagoscala/ (Meetings)
http://www.linkedin.com/in/deanwampler
http://www.polyglotprogramming.com
http://aquarium.rubyforge.org
http://www.contract4j.org

We have not really spent time to formalize what works and what doesn't
but most features do have tests. It is on my to-do list to come up
with a more formal spec of the language that is and the language we
want to see. Any who would like to start filling in blanks on the wiki
is welcome to do so.

- Charlie (mobile)

Hi --

On Wed, 28 Oct 2009, Charles Oliver Nutter wrote:

> On Tue, Oct 27, 2009 at 5:29 PM, Ryan Davis <ryand-ruby@zenspider.com> wrote:
>> You do NOT need static code for it to be fast. See smalltalk, self, and many
>> many other examples.
>
> Sure, it's just easier with static types. Don't like it, don't use it.

It seems like in this project, the notion of type as being synonymous
with class "wins", so to speak, over the non-class-bound concept of
type. I'm curious whether this is because doing it that way meshes
better with other languages that might be involved, or because the
concept of type as object capability at a given point in runtime is
(as I think it might be) almost impossible to capture statically at
all. I'm thinking of module inclusion as well as singleton-level
operations.


David

--
The          Ruby training with D. Black, G. Brown, J.McAnally
Compleat     Jan 22-23, 2010, Tampa, FL
Rubyist      http://www.thecompleatrubyist.com

David A. Black/Ruby Power and Light, LLC (http://www.rubypal.com)

From: "Ryan Davis" <ryand-ruby@zenspider.com>
> You do NOT need static code for it to be fast. See smalltalk, self,  
> and many many other examples.
> 
> Ruby's method dispatch is pathologically slow.

It would certainly be the best of all possible worlds
to have ruby code run two orders of magnitude faster
without giving any hints to the compiler.

In theory, then, should it truly be possible for the
ruby compiler to generate code that keeps floating
point values in registers, performs arithmetic on them,
and then passes the resulting values off to other
methods, without ever allocating the float objects on
ruby's heap?

(Or maybe such a thing would instead be achieved at
run-time through JIT'ing and sophisticated hotspot
analysis?)

Some of the more impressive Smalltalk benchmarks I've
been able to locate so far, still hedged a bit with
the following caveats:

http://www.cincomsmalltalk.com/userblogs/buck/blogView?showComments=true&entry=3354595110

"One more note about the Integer Arithmetic benchmark.
C# is smart enough to move memory variables to registers
at the start of the method and then just perform the
arithmetic in the registers. This isn't really typical
behavior in a real system where the arithmetic operations
are scattered throughout the system. In order to make it
more fair, I made the variables into instance variables
and marked them as volatile to force C# to fetch and
write them all the time. This makes the comparison to
Smalltalk much more fair."

That seems a bit disingenuous to me.  The advent of such
in-register optimizations is what finally allowed us to
stop using the 'register' keyword in C about 15 years ago.
Now it's implied we can toss that all aside in order to be
'fair' to Smalltalk?

I'm going to have to call shenanigans on that.

But anyway, if it's truly theoretically possible to
achieve C-like speeds in ruby without giving any hints to
the compiler then I'm all for it (obviously).

Otherwise, the remarkably unobtrusive type hinting Charlie
is experimenting with seems to me _vastly_ preferable to
the current situation of having to drop into C to get the
speed.  I don't *like* static typing.  But I like having
to drop from Ruby into C even less.

  *  *  *

I suppose the worry must be that there would be a slippery
slope, where if such static type hints existed in Ruby,
they would be overused, and we might end up with a
proliferation of third party gems and libraries which
were decidedly un-ruby-like.

Perhaps it would be possible for the compiler to always
generate two versions of any method which was declared with
type hints: the static version, and a purely dynamic
standard ruby version.  Then such type-hinted methods would
in effect provide an optional fast-path for users who were
in need of maximum speed, without imposing any un-ruby-like
restrictions on other users?

If that were possible, would there be any downside?  (An
honest question.  I'm not thinking of any myself, so far.)


Regards,

Bill

On Tue, Oct 27, 2009 at 8:01 PM, David A. Black <dblack@rubypal.com> 
wrote:
> It seems like in this project, the notion of type as being synonymous
> with class "wins", so to speak, over the non-class-bound concept of
> type. I'm curious whether this is because doing it that way meshes
> better with other languages that might be involved, or because the
> concept of type as object capability at a given point in runtime is
> (as I think it might be) almost impossible to capture statically at
> all. I'm thinking of module inclusion as well as singleton-level
> operations.

Yes, it was definitely a conscious decision to both use Java's type
system (which is immutable, or at least immutable enough that me as a
language implementer can just tell the JVM the class and method I want
to call at compile time), and explicit static typing (also a
requirement to be able to statically produce direct calls to specific
types and methods). The same could be mapped into any other backends
that have a notion of immutable types that are present at compile-time
rather than at runtime. In this case, it turns out that Ruby's syntax
presents at least the basic requirements necessary to produce JVM
bytecode for class and method structures like Java's types. Duby is
not yet able to replace Java, since it's missing many additional
features (field access from outside a class, annotations, generics,
enums, etc), but it is at least a fairly capable language in itself
that can call any Java type with no performance impact of any kind. In
essence, it's just a Ruby-lookalike for Java, but a Java somewhere pre
1.0. My plan is to have at least a Java 1.0 featureset for Duby 1.0.

The other goal with Duby is to produce a compiler that is extremely
pluggable. Most of the "keywords" like "implements" or "import" are
not hardcoded anywhere in the system; they are plugins loaded by the
compiler. Additional keywords can easily be added, or existing ones
could be removed. The language is fairly mutable.

The philosophy of Duby is simple. I wanted a language had the
syntactic elegance of Ruby, but the performance of and tight
integration with Java. At the same time, I didn't want to give up
metaprogramming-like capabilities, which is where the plugin system
comes from. You can do a surprising amount of metaprogramming just by
having a way to extend the compiler. And then you pay no runtime cost
for any of it.

Duby is certainly not intended to be a replacement for Ruby. There are
many things it will never support, like "eval", or interpreted
execution. It also will never ship with its own runtime
libraries...whatever the compiler produces needs to be "it", and you
are in control of what library dependencies you explicitly introduce
into your code. Writing a line of code should not suddenly force you
to ship a 2 or 4 or 20MB runtime library with your application.

I should also say it's fairly early days for Duby. The codebase needs
a lot of cleanup, the plugin system is fairly limited and cumbersome,
and error handling is not spectacular. And of course there are a lot
of Java/JVM features it can't support yet. But other than the parser
(JRubyParser) and the bytecode-generating backend (bitescript, which
wraps the Java ASM library), it's all written in Ruby. Hopefully folks
on this list will be interested in hacking on it.

FWIW, I'll be giving a talk on Duby at RubyConf (the "Ruby Mutants" 
talk).

- Charlie

On Tue, Oct 27, 2009 at 9:39 PM, Bill Kelly <billk@cts.com> wrote:
>
> (Or maybe such a thing would instead be achieved at
> run-time through JIT'ing and sophisticated hotspot
> analysis?)

Anything is possible. It's just really hard.

Even the best optimized dynamic language runtimes, many of which can
put values into registers and turn math into extremely fast low-level
native operations, still need to obey the semantics of the language.
In Ruby's case, that means you still need to ensure that the values
being passed into a numeric algorithm are actually Fixnums and make
sure that Fixnum itself has not been modified. If you don't check the
former, you may do optimized math against a non-math value. If you
don't do the former, Fixnum's mutability could lead to method
replacements (uncommon for math, admittedly) not being reflected in
already-optimized code. Performing those checks might be cheap, but
it's never free. In Duby, as in the JVM below it, if you're working
with a primitive int, you'll always be working with a primitive int.
Similar (but perhaps not as strong) guarantees apply to non-primitive
types.

> behavior in a real system where the arithmetic operations
> 'fair' to Smalltalk?
>
> I'm going to have to call shenanigans on that.

Shenanigans indeed! I've made the same sorts of claims myself, like
"JRuby is only a couple times slower than Java, /for equivalent
work/". That's my subtle way of saying "Java numeric algorithms that
work with all boxed 64-bit values and do virtual dispatches for all
math operations." It evens the playing field, but of course nobody
writing Java would write numeric algorithms performing virtual calls
against boxed 64-bit values. In my defense, however, math is one of
the few areas where JRuby will probably never be as fast as Java (or
Ruby implementations that don't need to use objects for Fixnums),
since our dynamic calls are almost as fast now as Java's virtual or
interface calls. If you're working with a bunch of non-numeric objects
in JRuby, we do very well.

> But anyway, if it's truly theoretically possible to achieve C-like speeds in
> ruby without giving any hints to
> the compiler then I'm all for it (obviously).

It's certainly possible to do much better than we have been doing.
MacRuby and Rubinius, for example, are now showing extremely good perf
numbers for numeric algorithms. In Rubinius's case, the recent JIT
work has started to make it conceivable that a pure-Ruby standard
library may be able to get within a few times of MRI's
implementations. And experiments with JRuby have been able to double
or triple our current Ruby performance, simply by making it easier for
the JVM to optimize Ruby code the same way it optimizes Java code. We
are able to approach both MacRuby and Rubinius even for numeric
algorithms where they have an advantage. It's an uphill battle--the
nature of Ruby is such that even our best efforts are littered with
performance-stealing typechecks--but we're all making great progress
and learning from each others' efforts.

Even if Ruby could run as fast as C (or Java, in my case) it would
still not fit the other requirements of Duby, like having no runtime
library or being able to present "real" Java classes and methods. So I
think even under the best circumstances, Duby (or something like it)
is still needed.

> Otherwise, the remarkably unobtrusive type hinting Charlie
> is experimenting with seems to me _vastly_ preferable to
> the current situation of having to drop into C to get the
> speed.  I don't *like* static typing.  But I like having
> to drop from Ruby into C even less.

Just as I find it ironic that I work all day, every day, implementing
Ruby...by writing in Java. It has led to a very solid, reasonably
performant implementation, but it would be a lot more fun to write in
something Ruby-like. If I can get much of what I like about Ruby by
using Duby and pay no performance penalty *at all*, I'd be very happy.
And you all might find it easier to contribute to JRuby, too :)

> in effect provide an optional fast-path for users who were
> in need of maximum speed, without imposing any un-ruby-like
> restrictions on other users?
>
> If that were possible, would there be any downside?  (An
> honest question.  I'm not thinking of any myself, so far.)

Interesting that you suggest this. I am also going to add dynamic
dispatch to Duby.

A second language project of mine, Surinx, is essentially Ruby syntax,
Java/JVM types, but all *dynamic* calls. It requires the dynamic
dispatch support coming up in Java 7, and it has a small runtime
library, but otherwise it is also written entirely in Ruby.
Performance is rather interesting with Surinx; it's much faster than
JRuby, since it doesn't have to deal with a mutable type system and it
lets the JVM optimized dynamic calls. For "equivalent work", Surinx is
only about 50% slower than Java, and that gap is likely to narrow as
the Hotspot engineers improve dynamic call optimzations.

But there's no real reason to keep the languages separate, so I intend
to merge them. The new language will be statically typed, except where
types cannot be statically determined (or perhaps where you explicitly
declare them as dynamic). If you write with no static types at all,
there will be very little to distinguish the syntax from plain old
Ruby (though of course it's still *not* Ruby, because it's still based
on Java's type system and class libraries).

Dean Wampler suggested the name "Dubious", which seems to fit its
dubious nature extremely well.

Surinx and Duby codebases are on my github account.

- Charlie

Dean,

On Tue, Oct 27, 2009 at 5:23 PM, Dean Wampler <deanwampler@gmail.com> 
wrote:
> Charlie,
>
> Your Strange Loop talk compared examples in Ruby, Duby, and Surinx. It was
> interesting how the examples differed very little from each other. I wonder
> if automated translation between the three would be possible? Perhaps the
> only realistic direction would be from Duby/Surinx -> Ruby (e.g., removing
> type information).


perhaps the Diamondback Ruby research would be applicable
to doing Ruby -> Duby/Surinx translation.



>> On Tue, Oct 27, 2009 at 5:13 PM, Bill Kelly <billk@cts.com> wrote:
>> cool.
>> know how to hack.
>> types and uses Java/JVM type system (though other backends are
> -  http://programmingscala.com
>
> twitter: @deanwampler, @chicagoscala
> Chicago-Area Scala Enthusiasts (CASE):
> -  http://groups.google.com/group/chicagoscala
> -  http://www.meetup.com/chicagoscala/ (Meetings)
> http://www.linkedin.com/in/deanwampler
> http://www.polyglotprogramming.com
> http://aquarium.rubyforge.org
> http://www.contract4j.org
>



--
thanks,
-pate
-------------------------
 Don't judge those who choose to sin differently than you do

http://on-ruby.blogspot.com
http://eldersjournal.blogspot.com

On Tue, Oct 27, 2009 at 6:23 PM, Dean Wampler <deanwampler@gmail.com> 
wrote:
> Charlie,
>
> Your Strange Loop talk compared examples in Ruby, Duby, and Surinx. It was
> interesting how the examples differed very little from each other. I wonder
> if automated translation between the three would be possible? Perhaps the
> only realistic direction would be from Duby/Surinx -> Ruby (e.g., removing
> type information).

Also...the fact that Duby/Surinx are not Ruby makes the translation
difficult to impossible. The code could translate, but almost none of
the API calls would. A translator from Duby/Surinx to/from Java, on
the other hand, would be much easier.

- Charlie

> perhaps the Diamondback Ruby research would be applicable
> to doing Ruby -> Duby/Surinx translation.

ruby2c also does some ruby "call analysis" then conversion to static 
types, if that's helpful.

Regarding it being impossible...I think for small inner loops you should 
be able to convert from ruby to duby or what not.  Plus duby is turing 
machine complete, right... ;)
-r

On Sat, Oct 31, 2009 at 5:06 PM, Roger Pack <rogerpack2005@gmail.com> 
wrote:
> ruby2c also does some ruby "call analysis" then conversion to static
> types, if that's helpful.

For a limited subset of Ruby (maybe the subset supported by Duby) it's
possible to globally infer some things. Not for all of Ruby, and
potentially not for the subset of Ruby syntax I want to support in
Duby.

> Regarding it being impossible...I think for small inner loops you should
> be able to convert from ruby to duby or what not.  Plus duby is turing
> machine complete, right... ;)

We are already planning this much with a new compiler infrastructure
that can locally infer and optimize loops and math and so on. It
should bring us a long way toward competing with Java or C for in-body
numeric logic.

- Charlie