Parse tree?

Ruby
1

I have a situation where it may make sense, although for some reason I
don’t want to, to use something like ParseTree to pull out each
complete Ruby expression and pass it to a block which will then
evaluate it in a particular context and make all kinds of magic
happen.

Is this a reasonable ParseTree use case?

@expressions_parse_tree_found.each |exp|
ArbitraryModule.module_eval(&exp)
end

–
Giles B.

Podcast: http://hollywoodgrit.blogspot.com
Blog: http://gilesbowkett.blogspot.com
Portfolio: http://www.gilesgoatboy.org
Tumblelog: http://giles.tumblr.com

2

Sorry, that was a dumb question, PT gives you sexps, not Ruby, you’d
need Ruby2Ruby for that, and even then it might not work.

–
Giles B.

Podcast: http://hollywoodgrit.blogspot.com
Blog: http://gilesbowkett.blogspot.com
Portfolio: http://www.gilesgoatboy.org
Tumblelog: http://giles.tumblr.com

3

Sorry, that was a dumb question, PT gives you sexps, not Ruby, you’d
need Ruby2Ruby for that, and even then it might not work.

why not?

As far as I can tell you get the whole shebang back, and I just need
particular expressions.

–
Giles B.

Podcast: http://hollywoodgrit.blogspot.com
Blog: http://gilesbowkett.blogspot.com
Portfolio: http://www.gilesgoatboy.org
Tumblelog: http://giles.tumblr.com

4

On Jan 2, 2008, at 22:01 , Giles B. wrote:

Sorry, that was a dumb question, PT gives you sexps, not Ruby, you’d
need Ruby2Ruby for that, and even then it might not work.

why not?

5

Giles B. wrote:

As far as I can tell you get the whole shebang back, and I just need
particular expressions.

Maybe you can use the ruby_parser gem?

6

On Jan 3, 2008, at 06:46 , Giles B. wrote:

Sorry, that was a dumb question, PT gives you sexps, not Ruby, you’d
need Ruby2Ruby for that, and even then it might not work.

why not?

As far as I can tell you get the whole shebang back, and I just need
particular expressions.

pt = ParseTree.new.parse_tree_for_string(“1+2”)
=> [[:call, [:lit, 1], :+, [:array, [:lit, 2]]]]

Ruby2Ruby.new.process(pt.first)
=> “(1 + 2)”

does that help?

7

On Jan 3, 2008, at 1:10 PM, Ryan D. wrote:

pt = ParseTree.new.parse_tree_for_string(“1+2”)
=> [[:call, [:lit, 1], :+, [:array, [:lit, 2]]]]
Ruby2Ruby.new.process(pt.first)
=> “(1 + 2)”

does that help?

ryan. even though i’ve seen that before i thought i’d add: that is
insanely cool!

a @ http://codeforpeople.com/

8

On Jan 3, 2008 1:33 PM, ara.t.howard [email protected] wrote:

It is really cool. I’ve been using R2R to build an auto-rspec thing
that re-creates the context of an expression so that you can test
against it. It’s actually quite fantastic.

Judson

9

Ruby2Ruby.new.process(pt.first)
=> “(1 + 2)”

does that help?

Sort of. It is obviously full of nifty and then some, and I think you
could use it to unit test metaprogramming, which is something the
metaprogramming-averse complain about the absence of.

But say I do this:

pt = ParseTree.new.parse_tree_for_string(“1 + 2 ; 1 * 2”)
(string):1: warning: useless use of + in void context
=> [[:block, [:call, [:lit, 1], :, [:array, [:lit, 2]]], [:call,
[:lit, 1], :, [:array, [:lit, 2]]]]]
Ruby2Ruby.new.process(pt.first)
=> “(1 + 2)\n(1 * 2)\n”

What if I just want 1 * 2?

–
Giles B.

Podcast: http://hollywoodgrit.blogspot.com
Blog: http://gilesbowkett.blogspot.com
Portfolio: http://www.gilesgoatboy.org
Tumblelog: http://giles.tumblr.com

10

It depends on what sort of ‘1 * 2’ you want… Do you want the last
expression in a block/defn/whatever? Like, are you trying to analyze
what the return type is? Or do you want to be aware of that void
context warning and discard the crap? Or do you simply want the second
thingy? Or do you want something entirely different that I am not
anticipating?

Well, I’m writing a code generator, and I thought I was very close to
finished, except I was basically doing this:

generated_code = Generators.module_eval(&block)

And that works great for tiny bits of code, it works at the statement
level, but when I want to just collect up all the statements in a
block, the whole thing kinds of goes south. So I have something which
generates all the individual statements perfectly but completely falls
apart when I want to create several statements. So it looks as if my
only option might be to rewrite everything, possibly adding a parser,
or using Ruby2C, or something like that. It’s kind of a pain in the
ass.

But if I have code like this:

single_line(statement)
statement(:which => spans,
:multiple => lines)

And I can turn it into an array of procs, or blocks, or even strings,
then I can just do

code_chunks.each {|code_chunk| @code <<
Generators.module_eval(code_chunk)}

Or something similar, and harness everything I have so far without
throwing it all away and starting over from scratch or close to it.
All I really need at that point is a collection of code chunks, in
some format which module_eval will be comfortable with.

=> s(:array, s(:lit, 2))

end
want and mess with it.
Very easy is a relative term. :slight_smile:

In practice none of the code I’m working with or expecting to work
with has more than one statement on a line, but obviously the whole
problem is that it has many statements in a block.

The thing is, I posted a few days ago about a problem with if, not
being able to override it, and if I’m using a fully-fledged parsing
library then I might as well undo my kludge and enable real support
for conditionals like if, unless, while, etc. But it seems like more
work than I’d intended. But I can’t deny that being able to use real
Ruby in the code which generates the other code would be a serious
improvement.

–
Giles B.

Podcast: http://hollywoodgrit.blogspot.com
Blog: http://gilesbowkett.blogspot.com
Portfolio: http://www.gilesgoatboy.org
Tumblelog: http://giles.tumblr.com

11

On Jan 3, 2008, at 21:06 , Giles B. wrote:

Well, I’m writing a code generator, and I thought I was very close to
finished, except I was basically doing this:

generated_code = Generators.module_eval(&block)

And that works great for tiny bits of code, it works at the statement
level, but when I want to just collect up all the statements in a
block, the whole thing kinds of goes south. So I have something which
generates all the individual statements perfectly but completely falls
apart when I want to create several statements.

Does this help?

spans, :multiple => lines)\n}"
There are clean ways to ensure it isn’t a proc as well.

12

On Jan 3, 2008, at 15:50 , Giles B. wrote:

[:lit, 1], :, [:array, [:lit, 2]]]]]

Ruby2Ruby.new.process(pt.first)
=> “(1 + 2)\n(1 * 2)\n”

What if I just want 1 * 2?

It depends on what sort of ‘1 * 2’ you want… Do you want the last
expression in a block/defn/whatever? Like, are you trying to analyze
what the return type is? Or do you want to be aware of that void
context warning and discard the crap? Or do you simply want the second
thingy? Or do you want something entirely different that I am not
anticipating?

There are a couple ways to go about it, depending on your answer
above. The easiest is Sexp/Array manipulation. Let’s say you were
interested in the arg list of the last call in the block:

sexp = Sexp.from_array pt.first
=> s(:block, s(:call, s(:lit, 1), :+, s(:array, s(:lit, 2))),
s(:call, s(:lit, 1), :*, s(:array, s(:lit, 2))))

sexp.last
=> s(:call, s(:lit, 1), :*, s(:array, s(:lit, 2)))

sexp.last.array
=> s(:array, s(:lit, 2))

This is a bad example because it has two calls in the block. With
unique sub-sexp types in a sexp, you can just pull them out by name:

sexp.call.array

But it balks when it is ambiguous.

But… if that isn’t what you want, there are more powerful means of
dealing with stuff:

class CallArgAnalyzer < SexpProcessor
def process_call sexp
sexp.shift # :call
recv = sexp.shift
name = sexp.shift
args = sexp.shift

 # do something with args

 return s(:nil) # or whatever... depends on context

end
end

CallArgAnalyzer.new.process(sexp.first)
=> s(:nil)

Granted, this doesn’t actually DO anything, but the framework to do
something is there and solid… Do note that SexpProcessor is meant
for transformational processing so it generally expects a sexp back
(you can set what the return type should be for all process_*
methods). The real point is that it is very easy to get at what you
want and mess with it.

13

On Jan 4, 2008 6:06 AM, Giles B. [email protected] wrote:

Well, I’m writing a code generator, and I thought I was very close to
finished, except I was basically doing this:

generated_code = Generators.module_eval(&block)

And that works great for tiny bits of code, it works at the statement
level, but when I want to just collect up all the statements in a
block, the whole thing kinds of goes south.

From the description you give, it sounds like you could just create a
new block that just calls a list of other blocks, and do ; separation?

Ie, if the line above is in a method called “foo(&block)”, call it
with something like

foo do
my_blocks.collect { |block| block.call + “;” }
end

Probably I’m not understanding your requirements, just thought I’d air
it in case it is this simple.

Eivind.

14

new block that just calls a list of other blocks, and do ; separation?

Ie, if the line above is in a method called “foo(&block)”, call it
with something like

foo do
my_blocks.collect { |block| block.call + “;” }
end

Probably I’m not understanding your requirements, just thought I’d air
it in case it is this simple.

The hard part is splitting an existing block into an array of blocks,
or strings, based on the logical division of statements; in other
words to say “turn this block into an array of statements” (in a way
that Ruby understands) is the tricky part.

–
Giles B.

Podcast: http://hollywoodgrit.blogspot.com
Blog: http://gilesbowkett.blogspot.com
Portfolio: http://www.gilesgoatboy.org
Tumblelog: http://giles.tumblr.com

15

On Jan 4, 2008, at 08:07 , Giles B. wrote:

That totally solves my problem. Or at least, if it’s guaranteed
to turn multi-line statements into one-liners every time, that very
probably totally solves my problem. (I haven’t tried it yet, I’ve just
proved it to be true, etc.) I can just pull it out of the proc{} with
a regex and then split it on the newlines. Baddabing baddaboom.

What is your 1-liner requirement for?

16

That totally solves my problem. Or at least, if it’s guaranteed
to turn multi-line statements into one-liners every time, that very
probably totally solves my problem. (I haven’t tried it yet, I’ve just
proved it to be true, etc.) I can just pull it out of the proc{} with
a regex and then split it on the newlines. Baddabing baddaboom.

What is your 1-liner requirement for?

Weird synch thing there. I plugged it in and it worked perfectly,
except that I’m splitting blocks now where I don’t need to. I can hack
around it by regexing on “do” but it’s non perfecto.

–
Giles B.

Podcast: http://hollywoodgrit.blogspot.com
Blog: http://gilesbowkett.blogspot.com
Portfolio: http://www.gilesgoatboy.org
Tumblelog: http://giles.tumblr.com

17

def code_for █ block.to_ruby; end
=> nil
code_for do
?> single_line(statement)
statement(:which => spans,
?> :multiple => lines)
end
=> “proc {\n single_line(statement)\n statement(:which =>
spans, :multiple => lines)\n}”

There are clean ways to ensure it isn’t a proc as well.

That totally solves my problem. Or at least, if it’s guaranteed
to turn multi-line statements into one-liners every time, that very
probably totally solves my problem. (I haven’t tried it yet, I’ve just
proved it to be true, etc.) I can just pull it out of the proc{} with
a regex and then split it on the newlines. Baddabing baddaboom.

–
Giles B.

Podcast: http://hollywoodgrit.blogspot.com
Blog: http://gilesbowkett.blogspot.com
Portfolio: http://www.gilesgoatboy.org
Tumblelog: http://giles.tumblr.com