# Lisper says to rubyist: gimmie your syntax tree

**URL:** <https://www.ruby-forum.com/t/lisper-says-to-rubyist-gimmie-your-syntax-tree/133990>\
**Category:** Ruby\
**Created:** [April 6, 2008, 4:40am UTC](https://www.ruby-forum.com/t/lisper-says-to-rubyist-gimmie-your-syntax-tree/133990 "2008-04-06T04:40:41Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![unknown](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/u/4bbf92/32.png) [@unknown](https://www.ruby-forum.com/u/unknown)\
**Post date:** [April 6, 2008, 4:40am UTC](https://www.ruby-forum.com/t/lisper-says-to-rubyist-gimmie-your-syntax-tree/133990/1 "2008-04-06T04:40:41Z")

</div>

module Tracer  
if $DEBUG  
def trace(str)  
STDERR.puts(str)  
end  
else  
def trace(str)  
end  
end  
end

class Application  
include Tracer

def initialize  
trace “initializing…”  
# …  
end

def run  
trace “running…”  
# …  
end  
end

I have unit tests to confirm things are working well, etc, however in  
complex multi-threaded code there’s no substitute for the trace output  
when problems arise. However even an empty method call is significant,

require ‘benchmark’  
include Tracer  
Benchmark.bm(16) { |b|  
n = 10\_000\_000  
b.report(“with trace”) { n.times { trace } }  
b.report(“without trace”) { n.times { } }  
}

```
                  user system total real

```

with trace 3.090000 0.000000 3.090000 ( 3.094240)  
without trace 1.010000 0.000000 1.010000 ( 1.015994)

So … why not …

# Manipulate the AST

SyntaxTree.each\_node { |node|  
if node.ast\_type == :method\_call and node.method\_name == “trace”  
node.ast\_type = :noop # or remove it, or whatever  
end  
}

# No more trace statements!

Application.new.run

Now I could gsub out those trace statements manually (or for a cleaner  
approach maybe ParseTree on RubyForge could be used). But that extra  
step is cumbersome compared to the magic I desire. If SyntaxTree were  
implemented, I think we would be close to having that bit of power  
which lisp provides but ruby does not. Removing trace statements  
would only be a trivial use of this Ultimate Power!

In my example SyntaxTree is using the code running in memory, but in  
principle I could say

code = SyntaxTree.entry\_point { Application.new.run }  
code.each\_node { |node|

# … manipulate the code …

}

# Execute some alternate-universe version of Application.new.run

code.call

Now THAT would be getting lispy.

---

<div class="post-metadata">

**Author:** ![unknown](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/u/4bbf92/32.png) [@unknown](https://www.ruby-forum.com/u/unknown)\
**Post date:** [April 6, 2008, 2:48pm UTC](https://www.ruby-forum.com/t/lisper-says-to-rubyist-gimmie-your-syntax-tree/133990/2 "2008-04-06T14:48:18Z")

</div>

removed\_email\_address@domain.invalid a écrit :

> So … why not …
> 
> # Manipulate the AST
> 
> SyntaxTree.each\_node { |node|  
> if node.ast\_type == :method\_call and node.method\_name == “trace”  
> node.ast\_type = :noop # or remove it, or whatever  
> end  
> }

I am also looking for something similar in Ruby (programNodeEnumerator  
of Smalltalk is so great !)

I’ve found these remarks :

-[Best Online Casino in Australia | Top Licensed Casinos for Gamblers](http://www.jroller.com/murphee/entry/ruby_let_s_get_an)  
-[http://r2.ifs.hsr.ch/rubyrefactoring.pdf](http://r2.ifs.hsr.ch/rubyrefactoring.pdf)

Hope this helps.  
JC

---

<div class="post-metadata">

**Author:** ![unknown](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/u/4bbf92/32.png) [@unknown](https://www.ruby-forum.com/u/unknown)\
**Post date:** [April 7, 2008, 7:35pm UTC](https://www.ruby-forum.com/t/lisper-says-to-rubyist-gimmie-your-syntax-tree/133990/3 "2008-04-07T19:35:39Z")

</div>

Tracer doesn’t inject “trace statements” into the AST. It calls  
set\_trace\_func, so there’s a callback for every event (method call,  
method return, new line, etc.).

The performance bottleneck of Tracer is that one of the arguments to the  
trace function is a binding; creating this binding is an expensive  
operation. AFAIK the only way to avoid this is to use the C API to call  
rb\_add\_event\_hook instead of using set\_trace\_func from Ruby. This is  
what the ruby-debug gem does, e.g.:

rdebug --trace test.rb

As for your idea of dynamically modifying program structure at run-time,  
I’ve toyed with this idea, but it’s not trivial. The biggest hurdle is  
Ruby’s handling of local/dynamic variables. IMO the AST is too  
low-level a program representation to be used for this. An intermediate  
representation that abstracts away some of the details of the AST is I  
think what’s needed.

Paul

---

<div class="post-metadata">

**Author:** ![unknown](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/u/4bbf92/32.png) [@unknown](https://www.ruby-forum.com/u/unknown)\
**Post date:** [April 8, 2008, 5:20pm UTC](https://www.ruby-forum.com/t/lisper-says-to-rubyist-gimmie-your-syntax-tree/133990/4 "2008-04-08T17:20:10Z")

</div>

On Tue, Apr 08, 2008 at 03:19:28AM +0900, Ryan D. wrote:

> I’m going this direction with the UnifiedRuby rewriter in ParseTree  
> (which I will probably extract into its own release), but it is  
> currently still too low level. I’m working on pulling it up a bit and  
> making it the default output from ParseTree and ruby\_parser. There  
> will still be “raw” versions you can call.

Nifty!

This will be very powerful when you are done. I wish I had the time…  
🙂

Paul

---

<div class="post-metadata">

**Author:** ![unknown](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/u/4bbf92/32.png) [@unknown](https://www.ruby-forum.com/u/unknown)\
**Post date:** [April 7, 2008, 8:20pm UTC](https://www.ruby-forum.com/t/lisper-says-to-rubyist-gimmie-your-syntax-tree/133990/5 "2008-04-07T20:20:25Z")

</div>

On Apr 7, 2008, at 10:34 , Paul B. wrote:

> An intermediate representation that abstracts away some of the  
> details of the AST is I think what’s needed.

Yup yup.

I’m going this direction with the UnifiedRuby rewriter in ParseTree  
(which I will probably extract into its own release), but it is  
currently still too low level. I’m working on pulling it up a bit and  
making it the default output from ParseTree and ruby\_parser. There  
will still be “raw” versions you can call.