Nexus Programming Language

Ruby
101

On Thu, Feb 26, 2009 at 8:53 AM, Chad P. [email protected]
wrote:

that
of what I said – that RPN parsers are easier to write than Polish
notation parsers.

My mistake, i thought you were saying the opposite of that. None the
less it
sounds like a challenge. I guess the easiest way would be to read tokens
into an array and then parse them as RPN, but that’s not very elegant
besides being a little harder. So i’ll come back to using a tree,
presumably
were using ruby since this is a ruby list so i would choose to implement
my
branch nodes as Hashs of the form;

{ :op => ‘+’, : lval => 7, :rval => 8 }

There are two ways to do it from here, but in both you read the tokens
creating a node if an operator is read or by populating it’s values, the
difference is whether you choose to evaluate a node when it’s complete
or
the whole expression when it is finished.
Is that more complex? I’m not really sure, i think i’ll write it up when
i
finish work.

102

David M. wrote:

Mike G. wrote:

I believe I’ve shown that either of the following is true:

Ruby can do everything Lisp can (ParseTree, etc)…

  • OR -
    there are things Ruby can do that Lisp cannot.

Uh, what? Where are you getting this?

In ruby, can you take a given method and make some of its arguments
lazily evaluated, without changing a thing in the program except the
method definition? Not even ParseTree will help you there. I rest my
case.

Incidentally, 1.8 is odd because it is one big AST in C, with no
compilation. Obviously 1.8 is very slow because of this. ParseTree
is just giving back what is already there. ParseTree is not available
in 1.9 or JRuby, nor is there a plan for it (and due to the nature of
compilation and optimization, I don’t think it will happen). MRI is a
prototype, implementation-wise.

In this formal, objective sense,
lisp is more powerful. Even if ruby greenspuns its own tools like
ParseTree, the language will still be different than the metalanguage.

Again, a syntactical difference.

Nope, not just a syntactical difference. In ruby, from a method you do
not have access to the caller’s argument expressions, even with
ParseTree. That is just one example.

Don’t take it personally. This fact is well understood. There is no
sense wiggling around it just to somehow respond with a
counter-argument.

Then try responding to that counter-argument, rather than “wiggling
around it” by simply repeating your own, and tacking on an argument from
authority.

Uh, what?

Well I’ve had enough experience to recognize incuriosity when I see
it. You are missing a specific piece of knowledge. If you were
curious, you could spend a few minutes reading about lisp macros in
order to gain some understanding of the issue at hand. But you want
to sit back and watch me “prove” something to you. No, I cannot make
you learn. It never works; you must learn for yourself.

I have already given sufficient context, and now I have provided a
specific case: lazy evaluation.

If you don’t want to learn, then that’s fine. But at least have the
good sense to recognize when you don’t know something.

103

Tony A. wrote:

On Tue, Feb 24, 2009 at 11:04 PM, Mike G.
[email protected]wrote:

The parentheses are there for power: code is data, data is code. When
evaluated as a language only, putting aside external issues such as
mindshare, it is an objective statement to say that lisp is more
powerful than any language which does not have the code-data
equivalence.

Homoiconicity doesn’t afford power so much as simplicity. Using parse
transformations in Ruby requires learning what the Ruby parser’s “sexps”
look like, whereas in Lisp you write your code directly in those sexps
so there is no disconnect.

No, it is more than convenience or simplicity. See below.

Once you have the Ruby parser’s sexps, the same class of transformations
are still available, as they are in any language which allows you to
programatically retrieve parser output and invoke the compiler from
within the language itself.

Aha! Now we are getting somewhere. How are you going to get the
sexps? You could convert your source to sexps, but then what? How
will you relate the sexps to the running code? If someone gives you a
Method instance, how will you find its sexp? Or if your sexp says how
Foo.bar was defined, how do you know the definition has not changed
during run-time? Or when you are inside a method, how can you get the
sexps of the argument expressions before they were passed?

However, for day-to-day programming tasks I find code in sexp form
harder to read (and write) than code in a more complex grammar which
is able to express the same idea with fewer tokens and less
syntactic noise.

Lisp is certainly better optimized for macros, but in my opinion
Ruby is better optimized for day-to-day use and readability.

I said earlier that I was leaving subjective issues aside. Lisp
programmers eventually do not see parens, so your subjective
assessment does not apply to them.

If you look at my posting history, you’ll see that I do ruby.

104

On Thu, Feb 26, 2009 at 12:03 AM, Mike G.
[email protected]wrote:

Incidentally, 1.8 is odd because it is one big AST in C, with no
compilation. Obviously 1.8 is very slow because of this. ParseTree
is just giving back what is already there. ParseTree is not available
in 1.9 or JRuby, nor is there a plan for it (and due to the nature of
compilation and optimization, I don’t think it will happen). MRI is a
prototype, implementation-wise.

Yes, you are correct, and I really wish ruby-core would embrace
ParseTree
and make it a standard feature of the language. The fact they haven’t
has
precluded alternative implementations like JRuby from implementing it.
There is JParseTree for JRuby but it isn’t actively maintained. And of
course YARV practically precludes a ParseTree implementation so there’s
little motivation for alternative implementations to even try to support
it. The future of Ruby seems to be one without ParseTree.

So Lisp 1, Ruby 0 I guess.

during run-time? Or when you are inside a method, how can you get the
sexps of the argument expressions before they were passed?

Conceeded, and given I accept the future of Ruby is one without
ParseTree
this argument is moot.

I said earlier that I was leaving subjective issues aside. Lisp
programmers eventually do not see parens, so your subjective
assessment does not apply to them.

Yes, however I would count them among the extreme minority of
programmers,
and would argue this is indicative of the fact that humans prefer
complex
grammars where more structure can be expressed by fewer tokens. Humans
have
really awesome brains which can infer meaning from ambiguous grammatical
constructions no computer possibly can (at least until computers become
sentient), so minimalistic grammars are really an insult to human
intelligence. But I guess there’s a certain class of Lisp weenie who
would
rather parse grammars like a computer :slight_smile:

105

2009/2/26 Eleanor McHugh [email protected]:

On 25 Feb 2009, at 21:54, Robert K. wrote:

What I probably wanted to say was: if you consider beauty as a criterion
for discussing languages it comes as no surprise that there is so little
agreement among people. :slight_smile:

A lot of these arguments though have less to do with beauty per se than they
do with an evangelical zeal to save the poor benighted heathens from their
dark and superstitious coding ways. Personally I like my superstitious ways
and I’m happy to leave benighted heathens to their own devices :slight_smile:

I see we seem to have a quite similar approach to those language
“discussions”. That’s why my engagement in them is usually brief.
There is no point in trying to “convert” someone of a different
“belief”.

I find the meta discussion about what
makes people have such strong feelings about particular positions and
why they want to convert others much more interesting. I guess it has
something to do with basic instincts of humans who need to know where
they “belong to”, i.e. what is their peer group and what are their
convictions. But this discussion is rarely seen in our forums; maybe
it has something to do with IT guys (they mostly are - unfortunately)
seeing themselves as rational people who believe there are rational
reasons for most or all of their decision. Which reminds me of
zealotry in political leftist circles who also believe(d) that they
know how history works and what it needs to make everyone happy. I
find it surprising that after Third Reich and Gulag there are still
people who believe that they are able to force happiness on
everyone…

PS: For me beauty in a program is clear structure and few but well chosen
lines of code.

The same here, and I like to believe that insofar as there are any objective
criterion for beauty in code those are the most easily expressed.

What a wonderful conclusion of a lengthy debate!

Warm regards

robert

106

Admittedly you would need to create an object or struct and manage
memory in
other languages, but here’s my take on polish notation in ruby.

class PN
#Cause why not? That’s why
attr_reader :result
def initialize
@result = 0
end

def op(token)
    #create the node saving the previous node if any
    @node = { :parent => @node, :op => token, :lval => nil}
end

def val(token)
    if(@node[:lval].nil?) then
        #If this is the first token then it is the lval
        @node[:lval] = token
    else
        #Otherwise the operation can be resolved
        case(@node[:op])
        when '+' then
            res = @node[:lval] + token
        when '-' then
            res = @node[:lval] - token
        when '/' then
            res = @node[:lval] / token
        when '*' then
            res = @node[:lval] * token
        end
        if(@node[:parent].nil?) then
            #This is the last(aka first) operation so save the

result
@result = res
@node = nil
else
#Send the result to the parent
@node = @node[:parent]
val(res)
end
end
end
end

32 * (10 + 5) + 3

pn = PN.new
token_list = [’+’, ‘*’, 32, ‘+’, 10, 5, 3]

token_list.each { |el|
if(el.is_a?(String)) then
pn.op(el)
else
pn.val(el)
end
}

puts pn.result

107

Mike G. wrote:

Well I’ve had enough experience to recognize incuriosity when I see
it. You are missing a specific piece of knowledge. If you were
curious, you could spend a few minutes reading about lisp macros in
order to gain some understanding of the issue at hand.

I have spent a few, before this discussion. I didn’t see much to
interest me – it seemed like a glorified preprocessor, so I wasn’t
terribly interested.

But…

I have already given sufficient context, and now I have provided a
specific case: lazy evaluation.

Yes, you have. I stand corrected – thank you.

108

On 26 Feb 2009, at 08:21, Robert K. wrote:

“discussions”. That’s why my engagement in them is usually brief.
There is no point in trying to “convert” someone of a different
“belief”.

I just like a good argument :wink: Plus I’ve always believed that the best
way to find out stuff is to make someone who already knows it so
defensively apoplectic that they give you their very best material in
defence. Lather, rinse, repeat :slight_smile:

know how history works and what it needs to make everyone happy. I
find it surprising that after Third Reich and Gulag there are still
people who believe that they are able to force happiness on
everyone…

Rationality has its uses, but carried to extremes it leaves the mind
trapped in a particular logical fallacies with no means of breaking
free. The universe after all is nothing more than a multi-dimensional
bumpy phase space full of highly misleading localised minima and
anyone who sits atop one is bound to be even more misguided than those
of us who get to the top of one slope and then decide to go for a
wander into the next valley. Who knows what exciting rabbit-holes we
may find there >8p

Ellie

Eleanor McHugh
Games With Brains
http://slides.games-with-brains.net

raise ArgumentError unless @reality.responds_to? :reason

109

Eleanor McHugh wrote:

There seems to be a fairly clean divide between the folks who like
language for being language, and those who like language for the
specific things it can do.

I would say, these aren’t completely unrelated – the grammar and
structure of a language, the way in which it lends itself to
communication, or to poetry, is one of the things a language can do.

And yet, part of the beauty of a language can be revealed in what it can
actually do. For example, I consider this to be beautiful:

get ‘/’ do
‘Hello, world!’
end

But while I’m sure someone can demonstrate far more elegant examples of
Sinatra code, and of code that Sinatra does better than other
frameworks, it might be difficult to show the full extent of its beauty
without the context that this is an http service, matching a request to
an intended response.

That was a bit of a ‘duh’ moment for me. I was still relatively new to
Rails and REST, but it was both aesthetically pleasing and immediately
obvious what it does – it was as if a full web stack hit my brain in an
instant.

I would say, dissolving beauty into separate ideas of form and function,
and then appreciating one without the other, is a bit like falling in
love with a photograph, never having met the person behind it. I am not
saying photographs cannot be beautiful, but if that is all you see in a
person, you are missing a lot.

We abandon the quest for beauty at our peril as it is the keenest
guide to correctness that we possess. Unfortunately an age of
aesthetic relativism has given it a bum wrap by attributing its
quality to things which patently aren’t beautiful and are far from
correct.

The most blatant reminder of this that I’ve seen was in this
presentation:

http://rubyhoedown2007.confreaks.com/session09.html

I won’t spoil it, though – anyone who hasn’t seen this, just find an
hour and watch it for yourself.

110

On Thu, Feb 26, 2009 at 02:10:59PM +0900, Martin DeMello wrote:

But don’t forget: Any sufficiently well-documented Lisp program
contains an ML program in the comments :slight_smile:

Nice. I hadn’t heard that one.

111

Mike G. wrote:

Method instance, how will you find its sexp?
That’s solvable – easy enough just to add an instance method to Method
objects. After all, we already have things like #arity, on Method and on
Proc objects.

Or if your sexp says how
Foo.bar was defined, how do you know the definition has not changed
during run-time?

Of course you don’t. How do you know this hasn’t happened in Lisp?

That’s like insisting that we can know that

class Foo
def bar
…
end
end

hasn’t changed. Of course it can:

Foo.send :remove_method, :bar

Or when you are inside a method, how can you get the
sexps of the argument expressions before they were passed?

Ah, that would be trickier, and depends very much on how much order of
execution is allowed to change.

I see no reason to believe it wouldn’t be possible with a slight
departure from current Ruby syntax – as an alternate way to accept
arguments. The question is whether that’s worth the potential confusion
– after all, we can already get explicit lazy evaluation easily enough
by passing a block instead of arguments.

For example, it’s possible to imagine a construct like this:

if foo(a = b)

do something with a

end

If a method is allowed to lazily evaluate its arguments – or not
evaluate them at all – I can’t count on ‘a’ being set, there, if ‘foo’
decides to do something strange. I’m guessing this kind of thing
wouldn’t come up much in practice, but I’m not sure.

112

Tony A. wrote:

On Thu, Feb 26, 2009 at 12:03 AM, Mike G. [email protected]wrote:

Incidentally, 1.8 is odd because it is one big AST in C, with no
compilation. Obviously 1.8 is very slow because of this. ParseTree
is just giving back what is already there. ParseTree is not available
in 1.9 or JRuby, nor is there a plan for it (and due to the nature of
compilation and optimization, I don’t think it will happen). MRI is a
prototype, implementation-wise.
Yes, you are correct, and I really wish ruby-core would embrace ParseTree
and make it a standard feature of the language. The fact they haven’t has
precluded alternative implementations like JRuby from implementing it.

Actually, I believe it is the other way round: embracing ParseTree
would preclude alternative implementations like JRuby from being
implemented in the first place.

Well, I admit, I was exaggerating. However, until recently, JRuby
had a (somewhat) ParseTree-compatible AST. The reason why they
decided to drop ParseTree-compatibility was simply that it would be
impossible to evolve the compiler while keeping the AST compatible.

ParseTree is intimately tied to the actual layout of the AST inside
MRI and to the actual inner workings of MRI. That would mean that the
alternative implementations would be intimately tied to the inner
workings of MRI – in other words, they wouldn’t actually be
“alternative” implementations any more, just clones.

This is especially bad, since MRI is actually a pretty marginal edge
case: it only supports Ruby 1.8, most other implementations support
either Ruby 1.9 or Ruby 1.8 and 1.9 in the same parser. MRI is the
only pure interpreter, every other implementation (and there’s about
a dozen of them) has a compiler – some even have only a compiler
and no interpreter at all. And, last but not least, even MRI’s creator
doesn’t consider it a very good Ruby Implementation.

JRuby is actually a pretty extreme countercase: where MRI’s parser and
AST are doing exactly one thing, JRuby’s parser and AST are used to
implement an interpreter, a dynamic JIT and a static ahead-of-time
batch compiler within the JRuby project, plus an interactive resident
background parser for syntax checking, syntax highlighting, code
formatting, code generation, code completion, static analysis, type
inference, code transformation and refactoring in at least 3 different
IDEs, and even to implement a compiler and type inference engine for
another language implementation (Duby).

There is JParseTree for JRuby but it isn’t actively maintained. And of
course YARV practically precludes a ParseTree implementation so there’s
little motivation for alternative implementations to even try to support
it. The future of Ruby seems to be one without ParseTree.

So Lisp 1, Ruby 0 I guess.

I wouldn’t say that. There are better ways than ParseTree. A
standardized generic, abstract, high-level parser API for example.

Countless man-hours and dollars, and even entire companies have been
lost in the battle for implementation-independent ASTs. There is a
reason why JVM bytecode and not ANDF rules the world. Let’s learn from
those mistakes.

jwm

113

On Thu, Feb 26, 2009 at 05:21:11PM +0900, Robert K. wrote:

zealotry in political leftist circles who also believe(d) that they
know how history works and what it needs to make everyone happy. I
find it surprising that after Third Reich and Gulag there are still
people who believe that they are able to force happiness on
everyone…

I think a more elegant schema would use:

<wander where="even more off-topic"></wandering>
114

On 26.02.2009 11:17, Eleanor McHugh wrote:

“discussions”. That’s why my engagement in them is usually brief.
There is no point in trying to “convert” someone of a different
“belief”.

I just like a good argument :wink:

Hm, ok then we do have a bit different approach.

Plus I’ve always believed that the best
way to find out stuff is to make someone who already knows it so
defensively apoplectic that they give you their very best material in
defence.

I see. But you do have to admit that it can be painful at times (not
necessarily for you but the peer). :slight_smile:

Lather, rinse, repeat :slight_smile:

LOL

know how history works and what it needs to make everyone happy. I
wander into the next valley. Who knows what exciting rabbit-holes we
may find there >8p

That’s another interesting way to look at it. May way to look at
decision making is this: rational decision making can only do “I choose
X because of Y”. Then you have to say “I choose Y because of Z”.
Basically the chain would not end unless at some point Y is something
which you simply do not question. It’s either a requirement coming from
the outside or it is a value that is part of your system of convictions.
In that sense rational decision making is a myth because all decisions
are rooted in the irrational. Hopefully I wasn’t too cryptic…

Kind regards

robert

115

On 26 Feb 2009, at 22:14, Robert K. wrote:

That’s another interesting way to look at it. May way to look at
decision making is this: rational decision making can only do “I
choose X because of Y”. Then you have to say “I choose Y because of
Z”. Basically the chain would not end unless at some point Y is
something which you simply do not question. It’s either a
requirement coming from the outside or it is a value that is part of
your system of convictions. In that sense rational decision making
is a myth because all decisions are rooted in the irrational.
Hopefully I wasn’t too cryptic…

It seems we’re both rationally irrational then as opposed to
irrationally rational lol

Ellie

Eleanor McHugh
Games With Brains
http://slides.games-with-brains.net

raise ArgumentError unless @reality.responds_to? :reason

116

Jörg W Mittag wrote:

Actually, I believe it is the other way round: embracing ParseTree
would preclude alternative implementations like JRuby from being
implemented in the first place.

Well, I admit, I was exaggerating. However, until recently, JRuby
had a (somewhat) ParseTree-compatible AST. The reason why they
decided to drop ParseTree-compatibility was simply that it would be
impossible to evolve the compiler while keeping the AST compatible.

Not impossible…there’s just been practically zero demand, limited
resources, and other things that needed more attention. I’m not opposed
to supporting PT in JRuby, but it’s never been a very high priority. And
now that ruby_parser can be used for almost all the offline PT-based
tools, there’s even less motivation.

ParseTree is intimately tied to the actual layout of the AST inside
MRI and to the actual inner workings of MRI. That would mean that the
alternative implementations would be intimately tied to the inner
workings of MRI – in other words, they wouldn’t actually be
“alternative” implementations any more, just clones.

I’ve also had an issue with PT basically just being MRI’s AST. We’ve
made numerous changes to the structure of the AST, some of them for very
good reasons. We’ve also unified a number of constructs that MRI still
keeps separate. In short, we’ve evolved, and while PT did a good job
exposing the general structure of MRI’s AST, it also exposes MRI’s
implementation warts. We don’t feel a strong need to emulate those
warts. But it’s doable :slight_smile: The fact that it hasn’t been done says more
about the demand for it than the challenge of making it happen.

I wouldn’t say that. There are better ways than ParseTree. A
standardized generic, abstract, high-level parser API for example.

ruby_parser handles some of this, but it’s another all-at-once parser;
you can’t stop and start it or handle subsections of code easily.
Really, it’s just a port of MRI’s Yacc/Bison parser to Racc.

  • Charlie
117

Charles Oliver N. wrote:

And now that ruby_parser can be used for almost all the offline
PT-based tools, there’s even less motivation.

Merb uses ParseTree for something called ActionArgs feature which, as I
understand it, actually relies on the name of function parameters. For
example, if you have an action which takes an argument called ‘foo’, and
that URL is hit with a querystring like ?foo=bar, it will pass that
value (‘bar’, in that case) as that argument.

This actually seems to no longer need ParseTree – that’s used on MRI –
but it also seems to have very different code for each of the three VMs
supported.

It’s worth mentioning that Merb is being merged into Rails. Merb 2 will
be Rails 3, and vice versa. So, while it’s solved, it’s not an entirely
obscure concern.

while PT did a good job exposing the general structure of MRI’s AST,
it also exposes MRI’s implementation warts. We don’t feel a strong
need to emulate those warts.

Ideally, the goal wouldn’t be to emulate MRI’s warts, but to expose
something generic enough to be common, but practical. I would hope it
would do a good job of exposing the original structure of the source
code, even if that is completely annihilated by optimizations in the
version actually executed.

In fact, I think the closest we have to “something like ParseTree” which
“helps support alternate implementations” might be Rubinius.
Unfortunately, I don’t seem to see any of the other alternate
implementations moving in that direction.

118

On Fri, Feb 27, 2009 at 1:52 AM, Chad P. [email protected]
wrote:

On Thu, Feb 26, 2009 at 02:10:59PM +0900, Martin DeMello wrote:

But don’t forget: Any sufficiently well-documented Lisp program
contains an ML program in the comments :slight_smile:

Nice. I hadn’t heard that one.

I’ve sadly lost the attribution, been trying to find it again for
years because I really would like to give credit to the genius who
came up with it.

martin

119 
<wander where="even more off-topic"></wandering>

                                      ^ Parse error,
                                        Wandering does not

compute.

120

David M. wrote:

Merb uses ParseTree for something called ActionArgs feature which, as I
understand it, actually relies on the name of function parameters. For
example, if you have an action which takes an argument called ‘foo’, and
that URL is hit with a querystring like ?foo=bar, it will pass that
value (‘bar’, in that case) as that argument.

This actually seems to no longer need ParseTree – that’s used on MRI –
but it also seems to have very different code for each of the three VMs
supported.

Well, in JRuby this is taking advantage of us being the most reflectable
Ruby implementation. Basically it’s a bit of Ruby code that accesses
JRuby internals directly to get at the method args. That approach works
reasonably well for small cases, but we’d probably want to expose our
AST to Ruby in a more direct way that could then be massaged into sexps.
JParseTree tried to do this, but without any JRuby runtime/API
help…I’m willing to provide that help, if someone wanted to give it
another go.

However there’s another complication with exposing the AST: sometimes we
don’t have one. For example, if a script is precompiled to JVM bytecode,
there’s no AST to present. And we would like to get to a point where JIT
compiling a method at runtime makes its associated AST eligible for GC,
since a substantial portion of memory in a typical Ruby app is taken up
by the AST itself. What do we do in those cases?

Ideally, the goal wouldn’t be to emulate MRI’s warts, but to expose
something generic enough to be common, but practical. I would hope it
would do a good job of exposing the original structure of the source
code, even if that is completely annihilated by optimizations in the
version actually executed.

Sure, I’m game for that. And with a little more revising, I think the
“unified parse tree” in later PT releases is pretty close. If we wanted
to do this right, we’d get the implementers together to agree what a
standard sexp-based representation of Ruby code should look like,
avoiding impl-specific warts and conveying exactly the information
needed.

In fact, I think the closest we have to “something like ParseTree” which
“helps support alternate implementations” might be Rubinius.
Unfortunately, I don’t seem to see any of the other alternate
implementations moving in that direction.

Rubinius is actually eliminating the intermediate sexps now because
going from parse to sexp to AST to compilation is just too slow. But
they’re still basically using MRI’s C parser, so PT is probably still
supportable in some capacity.

  • Charlie